-- Logs begin at Thu 2022-08-18 13:57:56 UTC, end at Thu 2022-08-18 15:09:52 UTC. -- Aug 18 14:37:08.647038 dv-cinder-852208-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 18 14:37:08.654229 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 18 14:37:08 2022] *** Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: nodename: dv-cinder-852208-7-iscsi Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: machine: x86_64 Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: clock source: unix Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: pcre jit disabled Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: detected number of CPU cores: 4 Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: current working directory: / Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: detected binary path: /usr/bin/uwsgi-core Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: your processes number limit is 47680 Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: your memory page size is 4096 bytes Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: detected max file descriptor number: 2048 Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: lock engine: pthread robust mutexes Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: thunder lock: enabled Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 18 14:37:08.657918 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 18 14:37:08.697296 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: Python main interpreter initialized at 0x5593c8f129c0 Aug 18 14:37:08.697296 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: python threads support enabled Aug 18 14:37:08.697296 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: your server socket listen backlog is limited to 100 connections Aug 18 14:37:08.697296 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: your mercy for graceful operations on workers is 90 seconds Aug 18 14:37:08.697652 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: mapped 403077 bytes (393 KB) for 2 cores Aug 18 14:37:08.697733 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: *** Operational MODE: preforking *** Aug 18 14:37:08.698353 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: *** uWSGI is running in multiple interpreter mode *** Aug 18 14:37:08.698353 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: spawned uWSGI master process (pid: 93165) Aug 18 14:37:08.698558 dv-cinder-852208-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 18 14:37:08.699614 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: spawned uWSGI worker 1 (pid: 93178, cores: 1) Aug 18 14:37:08.699614 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: spawned uWSGI worker 2 (pid: 93179, cores: 1) Aug 18 14:37:08.699614 dv-cinder-852208-7-iscsi devstack@keystone.service[93165]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 18 14:37:12.961209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 14:37:12.961209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: from cryptography.utils import int_from_bytes Aug 18 14:37:12.961958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 14:37:12.961958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: from cryptography.utils import int_from_bytes Aug 18 14:37:12.962378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 14:37:12.962378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: from cryptography.utils import int_from_bytes Aug 18 14:37:12.962486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 14:37:12.962486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: from cryptography.utils import int_from_bytes Aug 18 14:37:13.012862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 18 14:37:13.013026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 18 14:37:13.013107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 18 14:37:13.013180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 18 14:37:13.013241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] command line args: [] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 18 14:37:13.013381 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] command line args: [] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 18 14:37:13.013474 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 18 14:37:13.013631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ================================================================================ {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 18 14:37:13.013715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 18 14:37:13.013793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ================================================================================ {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 18 14:37:13.013864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] admin_token = **** {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.014040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] admin_token = **** {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.014120 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] config_dir = [] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.014272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.014350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] config_dir = [] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.014454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] config_source = [] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.014533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.014613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] config_source = [] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.014690 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] debug = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.014795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] debug = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.014881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.015068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.015155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.015221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.015360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.015438 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.015584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.015753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.015858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.015961 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.016070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] list_limit = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.016172 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] list_limit = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.016274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] log_config_append = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.016381 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] log_config_append = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.016483 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.016580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.016675 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] log_dir = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.016774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] log_dir = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.016875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] log_file = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.016981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] log_file = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.017077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.017200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.017278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.017399 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.017483 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.017563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.017687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.017766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.017885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.017970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.018051 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.018176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.018253 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.018375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.018451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.018581 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.018669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.018742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.018866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.018942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.019086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.019167 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.019315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.019437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.019568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.019724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.019833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.019911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.020104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.020225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.020590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.020710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.020835 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] publish_errors = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.020945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] publish_errors = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.021065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.021180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.021180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.021398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.021512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.021625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.021738 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.021874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.021998 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.022119 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.022240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] standard_threads = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.022360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] standard_threads = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.022489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.022605 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.022726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.022853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.022976 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.023091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.023256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] use_journal = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.023379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] use_journal = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.023379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] use_json = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.023601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] use_json = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.023735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] use_stderr = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.023853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] use_stderr = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.023981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] use_syslog = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.024098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] use_syslog = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.024212 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.024326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 14:37:13.024446 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.024561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.024688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.024811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.024925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.025041 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.025155 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.025269 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.025391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.025507 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.025621 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.025742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.025857 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.025971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.026094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.026217 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.026338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.026454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.026568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.026683 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.026806 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.026928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.027048 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.027164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.027278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.027392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.027524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.027651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.027783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.027899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.028003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.028117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.028248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.028365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.028480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.028595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.028709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.028900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.029018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.029138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.029261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.029376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.029498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.029622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.029738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.029852 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.029966 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.030087 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.030203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.030330 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.030453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.030568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.030681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.030796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.030910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.031033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.031148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.031264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.031385 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.031500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.031615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.031759 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.031878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.031998 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.032113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.032234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.032350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.032477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.032604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.032721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.032836 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.032950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.033070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.033194 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.033309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.033425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.034024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.034145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.034260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.034383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.034500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.034614 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.034727 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.034841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.034955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.035145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.035262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.035383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.035499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.035613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.035771 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.035903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.036026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.036142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.036256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.036354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.036450 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.036554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.036650 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.036752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.036855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.036952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.037047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.037150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.037248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.037364 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.037480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.037596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.037718 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.037835 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.037955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.038066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.038206 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.038329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.038458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.038575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.038692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.038807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.038922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.039037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.039161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.039267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.039380 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.039495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.039615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.039761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.039889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.040014 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.040143 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.040263 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.040386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.040502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.040620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.040736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.040862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.040985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.041101 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.041217 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.041340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.041456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.041579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.041695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.041813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.041928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.042049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.042166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.042289 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.042404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.042519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.042634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.042755 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.042877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.043001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.043117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.043238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.043355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.043475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.043591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.043769 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.043890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.044006 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.044121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.044236 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.044358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.044473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.044588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.044703 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.044826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.044943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.045059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.045183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.045300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.045415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.045530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.045644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.045759 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.045883 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.046000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.046115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.046230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.046344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.046466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.046652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.046770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.046885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.047001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.047116 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.047244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.047368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.047496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.047613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.047741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.047858 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.047984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.048108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.048233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.048356 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.048472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.048587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.048702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.048817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.048940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.049057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.049173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.049894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.050021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.050140 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.050268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.050389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] auth.external = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.050504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] auth.external = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.050620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.050743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.050866 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.051017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.051144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.051271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.051389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] auth.password = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.051505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] auth.password = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.051621 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] auth.token = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.051763 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] auth.token = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.051882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.052014 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.052133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.052259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.052366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.052480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.052597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.052737 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.052908 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.053047 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.053164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.053282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.053398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.053524 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.053647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] credential.caching = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.053765 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] credential.caching = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.053880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.053997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.054112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.054243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.054369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.054486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.054602 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.054725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.054842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.054967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.055085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.055201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.055316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.055434 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.055550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.055688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.055809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.055926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.056043 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.056169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.056294 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.056420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.056539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.056654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.056770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.056886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.057008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.057134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.057251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.057368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.057485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] federation.caching = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.057601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] federation.caching = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.057717 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.057843 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.057960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.058077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.058199 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.058317 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.058452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.058572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.058694 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.058811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.058928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.059044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.059169 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.059287 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.059403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.059519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.059636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.059798 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.059932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.060057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.060174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.060290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.060417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.caching = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.060535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.caching = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.060660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.060779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.060895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.061011 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.061128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.061244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.061436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.061556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.061674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.061790 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.061906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.062022 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.062150 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.062268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.062384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.062500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.062616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.062732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.062860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.062979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.063095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.063212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.063327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.063443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.063568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.063703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.063823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.063939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.064055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.064171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.064300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.064419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.064525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.064640 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.064763 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.064881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.065008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.065126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.065242 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.065359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.065475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.065590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.065714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.065832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.065949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.066065 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.066180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.066296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.066421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.066538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.066654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.066770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.066886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.067011 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.067129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.067245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.067361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.067477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.067593 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.067780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.067902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.068019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.068136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.068252 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.068368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.068497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.068616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.068732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.068849 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.068966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.069084 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.069208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.069326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.069442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.069559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.069675 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.069791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.069916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.070034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.070152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.070274 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.070394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.070511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.070636 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.070754 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.070870 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.070987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.071105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.071221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.071345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.071463 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.071579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.071710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.071828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.071945 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.072077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.072197 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.072314 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.072430 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.072546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.072669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.072795 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.072913 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.073030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.073146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.073264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.073380 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.073506 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.073631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.073749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.073866 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.073982 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.074099 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.074224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.074343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.074459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.074576 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.074692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.074808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.074933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.075052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.075169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.075286 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.075403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.075519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.075643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.075793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.075910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.076027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.076143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.076259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.076389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.076508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.076625 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.076741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.076858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.076974 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.077098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.077216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.077332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.077450 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.077566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.077683 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.077807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.077925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.078043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.078159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.078275 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.078391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.078516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.078634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.078751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.078867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.078983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.079099 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.079224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.079342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.079459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.079576 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.079711 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.079830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.079959 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.080079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.080195 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.080312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.080433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.080551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.080677 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.080795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.080912 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.081029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.081146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.081262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.081392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.081510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] resource.caching = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.081628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] resource.caching = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.081744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.081860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.081976 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.082102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.082220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.082336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.082458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.082576 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.082702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.082820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.082937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.083054 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.083170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.083287 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.083411 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.083529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.083645 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.083808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.083925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] role.caching = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.084041 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] role.caching = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.084233 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] role.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.084353 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] role.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.084470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.084586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.084702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.084820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.084949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.085068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.085185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.085302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.085419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.085536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.085661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.085781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.085898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.086014 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.086130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.086247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.086372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.086490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.086606 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.086723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.086840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.086956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.087081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.087199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.087316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.087433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.087550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.087666 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.087815 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.087935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.088052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.088168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.088285 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.088401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.088527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.088645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.088761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.088877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.088995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.089120 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.089239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.089355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.089472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.089643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.089763 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.089889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.090008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.090125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.090242 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.090362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.090480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.090605 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.090724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.090840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.090964 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.091084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.091200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.091325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.091443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.091560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.091728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.091850 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.091967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.092096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.092215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.092331 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.092447 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] token.caching = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.092563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] token.caching = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.092679 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.092804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.092922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.093038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.093155 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.093271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.093387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.093512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.093630 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.093746 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.093862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.093978 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.094095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.094220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.094338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.094454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.094570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.094686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.094803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.094931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.095050 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.095166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.095283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.095400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.095515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.095640 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.095774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.095923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.096530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.096652 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.096769 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.096897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.097016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.097133 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.097249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.connection = **** {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.097364 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.connection = **** {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.097494 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.097621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.097739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.097856 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.097981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.098099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.098216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.098332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.098483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.098603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.098728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.098846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.098963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.099080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.099196 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.099313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.099439 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.099557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.099674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.099823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.099940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.100056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.100184 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.100303 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.100424 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.100542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.100659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.100775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.100900 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.101018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.101134 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.101250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.101373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.101490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.101616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.101734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 14:37:13.101850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93179) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 18 14:37:13.101967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 18 14:37:13.126667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.126896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.127122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.127207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.127430 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.127655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.135442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.136118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.163945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.164056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.164134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.164274 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.217591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.218211 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.266340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.266570 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.267013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.267249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.422994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.423166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.423314 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.423398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.423494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.423573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.423649 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.423772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.423850 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.423940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.424018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.424094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.447126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.447360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.447454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.447572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.447656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.447820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.447903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.448040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.448141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.448251 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.448333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.448461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.448543 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.448673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.460091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.460230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.472729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.472956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.473061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.473181 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.473264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.473402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.473484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.473618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.473709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.473832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.473915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.474043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.474126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.474251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93179) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 14:37:13.491069 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.491284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 14:37:13.491394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.491606 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 14:37:13.491711 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.491842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 14:37:13.491971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.492122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.492205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 14:37:13.492385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.492475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.492682 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.492896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.493163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.493242 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.493437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.502726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.503006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.503088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.503277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.503359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.503563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.503645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.503870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.510493 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.511084 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.513389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.513677 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.513957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.514038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.514236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.514318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.514514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.514596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.514774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.514855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.515046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.515136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.515310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.515391 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.515585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.515710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.515885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.515993 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.516264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.516344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.516540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.516819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.540160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.540324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.541619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.541774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.541903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.542058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.546546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.546727 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.546834 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.547010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.547105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.547301 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.553455 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.553693 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.553777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.553977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.557788 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.558060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.558143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.558348 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.558429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.558627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.558710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.558896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.558977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.559168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.559258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.559437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.559519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.559723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.559806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.559995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.577024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.577310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.577448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.577728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.581698 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.582103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.584929 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.585228 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.585310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.585605 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.589451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.589807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.592603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.592960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.595834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.596667 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.596758 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.596957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.597039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.597242 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.597333 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.597520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.597611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.597801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.597883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.598092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.613855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.614017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.614138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.614307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.614419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.614578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.614686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.614855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.621572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.621748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.623353 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.623520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.623631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.623821 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.626312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.626462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.626581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.626746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.626867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.627028 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.627167 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.627867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.627999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.628154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.639246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.639528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.639833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.639919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.640115 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.640198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.640396 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.640489 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.640674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.640758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.640948 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.641030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.641223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.641502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.656366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.656922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.659939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.660507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.662806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.663370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.664279 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.664832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.665743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.666025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.666300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.666383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.666585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.666881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.672817 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.673428 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.674838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.675452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.677815 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.678110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.678412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.678494 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.678694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.678782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.679017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.679296 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.686624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.686915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.687194 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.687278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.687523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.687836 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.694131 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.694414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.694696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.694790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93179) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 14:37:13.694974 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.695066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.695248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.695339 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.695520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.695611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.695819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.695903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.696101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.696183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.696381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.696464 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.696650 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.696769 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.696929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.697019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.697191 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.697281 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.697465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.697594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.697727 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.697808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.698324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.698609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93179) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 14:37:13.731881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5593c8f129c0 pid: 93178 (default app) Aug 18 14:37:13.733016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5593c8f129c0 pid: 93179 (default app) Aug 18 14:37:13.735221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95809d-22f8-4857-8f1e-c7d0e5652f25 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:13.735506 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95809d-22f8-4857-8f1e-c7d0e5652f25 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:13.735802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95809d-22f8-4857-8f1e-c7d0e5652f25 None None] PATH_INFO: `/v3` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:13.740511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Thu Aug 18 14:37:13 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:20.246027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e251-2239-4bc9-b88c-23af08d94437 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:20.246536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e251-2239-4bc9-b88c-23af08d94437 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:20.246619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e251-2239-4bc9-b88c-23af08d94437 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:20.251047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:20.256699 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c0212-7b4e-42c3-bf17-95c0e811226c None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:20.257107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c0212-7b4e-42c3-bf17-95c0e811226c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:20.257247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c0212-7b4e-42c3-bf17-95c0e811226c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:20.339659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: INFO dbcounter [None req-c01c0212-7b4e-42c3-bf17-95c0e811226c None None] Registered counter for database keystone Aug 18 14:37:20.361015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG oslo_db.sqlalchemy.engines [None req-c01c0212-7b4e-42c3-bf17-95c0e811226c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93178) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 18 14:37:20.367750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writer thread running {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 18 14:37:20.547416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG passlib.handlers.bcrypt [None req-c01c0212-7b4e-42c3-bf17-95c0e811226c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=93178) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 18 14:37:20.547786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG passlib.handlers.bcrypt [None req-c01c0212-7b4e-42c3-bf17-95c0e811226c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93178) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 18 14:37:20.603704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-c01c0212-7b4e-42c3-bf17-95c0e811226c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:20.605306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c01c0212-7b4e-42c3-bf17-95c0e811226c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:20.730191 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 18 14:37:21.177108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66e251-2239-4bc9-b88c-23af08d94437 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:21.245528 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: INFO dbcounter [None req-ff66e251-2239-4bc9-b88c-23af08d94437 None None] Registered counter for database keystone Aug 18 14:37:21.267516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG oslo_db.sqlalchemy.engines [None req-ff66e251-2239-4bc9-b88c-23af08d94437 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93179) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 18 14:37:21.274195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writer thread running {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 18 14:37:21.393767 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66e251-2239-4bc9-b88c-23af08d94437 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:21.398072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b44e87-7813-4150-a8c7-df46d5d96cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:21.398700 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b44e87-7813-4150-a8c7-df46d5d96cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:21.398981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b44e87-7813-4150-a8c7-df46d5d96cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:21.399255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b44e87-7813-4150-a8c7-df46d5d96cc2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:21.407280 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b44e87-7813-4150-a8c7-df46d5d96cc2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:21.459970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: WARNING keystone.common.rbac_enforcer.enforcer [None req-37b44e87-7813-4150-a8c7-df46d5d96cc2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 18 14:37:21.460259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b44e87-7813-4150-a8c7-df46d5d96cc2 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:21.469575 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Thu Aug 18 14:37:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:37:21.475510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c0212-7b4e-42c3-bf17-95c0e811226c None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:21.490389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c0212-7b4e-42c3-bf17-95c0e811226c None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:21.494822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b05ad6-9ab5-4344-aaab-6e90981bf9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:21.495952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b05ad6-9ab5-4344-aaab-6e90981bf9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:21.496241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b05ad6-9ab5-4344-aaab-6e90981bf9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:21.496515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b05ad6-9ab5-4344-aaab-6e90981bf9a9 admin admin] PATH_INFO: `/v3/projects/fa81e49fae72473b99b445418d4fe5de` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:21.498356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b05ad6-9ab5-4344-aaab-6e90981bf9a9 admin admin] RBAC: Authorizing `identity:get_project(project_id=fa81e49fae72473b99b445418d4fe5de)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:21.551857 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: WARNING keystone.common.rbac_enforcer.enforcer [None req-08b05ad6-9ab5-4344-aaab-6e90981bf9a9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 18 14:37:21.552198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b05ad6-9ab5-4344-aaab-6e90981bf9a9 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:21.556707 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Thu Aug 18 14:37:21 2022] GET /identity/v3/projects/fa81e49fae72473b99b445418d4fe5de => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:23.435055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047bc3d-e3c3-41a4-8eb8-94bb8b8cf7fd None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:23.435625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047bc3d-e3c3-41a4-8eb8-94bb8b8cf7fd None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:23.435625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047bc3d-e3c3-41a4-8eb8-94bb8b8cf7fd None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:23.439653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:23.444959 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324fd85-b420-45ee-9c5b-d4f3172ced99 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:23.445354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324fd85-b420-45ee-9c5b-d4f3172ced99 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:23.445521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324fd85-b420-45ee-9c5b-d4f3172ced99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:23.465053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-9324fd85-b420-45ee-9c5b-d4f3172ced99 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:23.466569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9324fd85-b420-45ee-9c5b-d4f3172ced99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:23.474904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 18 14:37:23.740492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047bc3d-e3c3-41a4-8eb8-94bb8b8cf7fd None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:23.752544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047bc3d-e3c3-41a4-8eb8-94bb8b8cf7fd None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:23.756674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9d294-d130-446e-a625-5d1244264596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:23.757260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9d294-d130-446e-a625-5d1244264596 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:23.757536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9d294-d130-446e-a625-5d1244264596 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:23.757808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9d294-d130-446e-a625-5d1244264596 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:23.765216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9d294-d130-446e-a625-5d1244264596 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:23.766987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9d294-d130-446e-a625-5d1244264596 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:23.775963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Thu Aug 18 14:37:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:37:23.781057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324fd85-b420-45ee-9c5b-d4f3172ced99 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:23.790624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324fd85-b420-45ee-9c5b-d4f3172ced99 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:23.794929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d998779-1cd8-4f9e-8740-12dca13f004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:23.795547 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d998779-1cd8-4f9e-8740-12dca13f004c admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:23.795863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d998779-1cd8-4f9e-8740-12dca13f004c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:23.796143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d998779-1cd8-4f9e-8740-12dca13f004c admin admin] PATH_INFO: `/v3/users/5064296cd28f403599a02675cb0c7103` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:23.797451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d998779-1cd8-4f9e-8740-12dca13f004c admin admin] RBAC: Authorizing `identity:get_user(user_id=5064296cd28f403599a02675cb0c7103)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:23.799296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d998779-1cd8-4f9e-8740-12dca13f004c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:23.800846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Thu Aug 18 14:37:23 2022] GET /identity/v3/users/5064296cd28f403599a02675cb0c7103 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:25.761683 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d8e41-ba34-4aec-95af-06ea0f7d1a38 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:25.762228 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d8e41-ba34-4aec-95af-06ea0f7d1a38 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:25.762228 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d8e41-ba34-4aec-95af-06ea0f7d1a38 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:25.763551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:37:25.763824 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2c4a4-1f05-4ef2-b201-07910de252b8 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:25.764241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2c4a4-1f05-4ef2-b201-07910de252b8 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:25.764378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2c4a4-1f05-4ef2-b201-07910de252b8 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:25.765781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:25.769055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa84ac5-3147-4a17-b15c-34daf67fedf3 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:25.769340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa84ac5-3147-4a17-b15c-34daf67fedf3 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:25.769618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa84ac5-3147-4a17-b15c-34daf67fedf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:25.771377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492101c-f0d3-4f28-ae7e-46aef6936391 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:25.771658 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492101c-f0d3-4f28-ae7e-46aef6936391 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:25.771966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492101c-f0d3-4f28-ae7e-46aef6936391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:25.789566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-0492101c-f0d3-4f28-ae7e-46aef6936391 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:25.790745 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0492101c-f0d3-4f28-ae7e-46aef6936391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:25.810746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG passlib.handlers.bcrypt [None req-3aa84ac5-3147-4a17-b15c-34daf67fedf3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=93179) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 18 14:37:25.811158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG passlib.handlers.bcrypt [None req-3aa84ac5-3147-4a17-b15c-34daf67fedf3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93179) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 18 14:37:25.857334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-3aa84ac5-3147-4a17-b15c-34daf67fedf3 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:25.858919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3aa84ac5-3147-4a17-b15c-34daf67fedf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:25.870871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 7/12] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:37:25.957828 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 6/13] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:26.136212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492101c-f0d3-4f28-ae7e-46aef6936391 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:26.159117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:26.230214 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa84ac5-3147-4a17-b15c-34daf67fedf3 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:26.241570 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492101c-f0d3-4f28-ae7e-46aef6936391 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:26.275069 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247cb77-628e-4abc-ac5c-6b710974b93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:26.275670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247cb77-628e-4abc-ac5c-6b710974b93e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:26.275980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247cb77-628e-4abc-ac5c-6b710974b93e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:26.276254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247cb77-628e-4abc-ac5c-6b710974b93e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:26.331490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa84ac5-3147-4a17-b15c-34daf67fedf3 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:26.333715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:26.341831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247cb77-628e-4abc-ac5c-6b710974b93e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:26.343325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247cb77-628e-4abc-ac5c-6b710974b93e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:26.368085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:26.369975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a5d7b-d4f1-45b1-a8a4-3ebf04913826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:26.371108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a5d7b-d4f1-45b1-a8a4-3ebf04913826 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:26.371459 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a5d7b-d4f1-45b1-a8a4-3ebf04913826 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:26.371653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a5d7b-d4f1-45b1-a8a4-3ebf04913826 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:26.383269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a5d7b-d4f1-45b1-a8a4-3ebf04913826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-0d7a5d7b-d4f1-45b1-a8a4-3ebf04913826 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:26.391619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:26.392568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:37:26.393604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:37:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:37:26.398413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a5d7b-d4f1-45b1-a8a4-3ebf04913826 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:26.452146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 8/15] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:37:26.457596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247cb77-628e-4abc-ac5c-6b710974b93e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:26.494509 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a5d7b-d4f1-45b1-a8a4-3ebf04913826 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:26.526795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959c681-7b40-44ed-a8e3-bbce17e5659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:26.527391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959c681-7b40-44ed-a8e3-bbce17e5659a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:26.527665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959c681-7b40-44ed-a8e3-bbce17e5659a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:26.527963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959c681-7b40-44ed-a8e3-bbce17e5659a None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:26.528613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959c681-7b40-44ed-a8e3-bbce17e5659a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:26.528912 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959c681-7b40-44ed-a8e3-bbce17e5659a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:26.530342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959c681-7b40-44ed-a8e3-bbce17e5659a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:26.542165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 8/16] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:26.547565 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959c681-7b40-44ed-a8e3-bbce17e5659a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:26.553612 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247cb77-628e-4abc-ac5c-6b710974b93e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:26.573616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:26.576067 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:26.587185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9b062-666d-49ca-91d9-e5a467aad46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:26.587846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b062-666d-49ca-91d9-e5a467aad46e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:26.588154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b062-666d-49ca-91d9-e5a467aad46e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:26.588429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b062-666d-49ca-91d9-e5a467aad46e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:26.598657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9b062-666d-49ca-91d9-e5a467aad46e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:26.600169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9b062-666d-49ca-91d9-e5a467aad46e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-89f9b062-666d-49ca-91d9-e5a467aad46e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:37:26.618084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:26.619184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:37:26.620266 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:37:26.620266 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:37:26.620266 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:37:26.620266 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:37:26.627988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9b062-666d-49ca-91d9-e5a467aad46e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:26.645396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959c681-7b40-44ed-a8e3-bbce17e5659a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:26.679170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f4d5b-8f22-479a-b0fb-1cc6aa42a057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:26.679834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4d5b-8f22-479a-b0fb-1cc6aa42a057 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:26.680114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4d5b-8f22-479a-b0fb-1cc6aa42a057 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:26.680418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4d5b-8f22-479a-b0fb-1cc6aa42a057 None admin] PATH_INFO: `/v3/users/5064296cd28f403599a02675cb0c7103` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:26.681891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f4d5b-8f22-479a-b0fb-1cc6aa42a057 None admin] RBAC: Authorizing `identity:get_user(user_id=5064296cd28f403599a02675cb0c7103)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:26.683336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f4d5b-8f22-479a-b0fb-1cc6aa42a057 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:26.685090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 9/18] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:26 2022] GET /identity/v3/users/5064296cd28f403599a02675cb0c7103 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:26.690423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f4d5b-8f22-479a-b0fb-1cc6aa42a057 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:26.725583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9b062-666d-49ca-91d9-e5a467aad46e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:26.758158 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b4372-2313-4d28-855f-340cefd9ef7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:26.758750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b4372-2313-4d28-855f-340cefd9ef7c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:26.759032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b4372-2313-4d28-855f-340cefd9ef7c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:26.759302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b4372-2313-4d28-855f-340cefd9ef7c None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:26.759966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b4372-2313-4d28-855f-340cefd9ef7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:26.760268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b4372-2313-4d28-855f-340cefd9ef7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:26.761643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b4372-2313-4d28-855f-340cefd9ef7c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:26.778591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 10/19] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:26.781941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:26.790656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f4d5b-8f22-479a-b0fb-1cc6aa42a057 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:26.827299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83eeca7-a8ed-40c5-a21c-1d49de416264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:26.827994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eeca7-a8ed-40c5-a21c-1d49de416264 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:26.828273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eeca7-a8ed-40c5-a21c-1d49de416264 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:26.828545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eeca7-a8ed-40c5-a21c-1d49de416264 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:26.829756 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83eeca7-a8ed-40c5-a21c-1d49de416264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:26.831776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83eeca7-a8ed-40c5-a21c-1d49de416264 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:26.833363 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 10/20] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:26.839492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b4372-2313-4d28-855f-340cefd9ef7c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:26.877762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:26.948581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b4372-2313-4d28-855f-340cefd9ef7c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:26.983398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151d7bb-f767-4a8d-b2fc-5753bf00453f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:26.984031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151d7bb-f767-4a8d-b2fc-5753bf00453f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:26.984339 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151d7bb-f767-4a8d-b2fc-5753bf00453f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:26.984607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151d7bb-f767-4a8d-b2fc-5753bf00453f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:26.985329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151d7bb-f767-4a8d-b2fc-5753bf00453f None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.domain.id=default, user.id=5064296cd28f403599a02675cb0c7103) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:26.985631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151d7bb-f767-4a8d-b2fc-5753bf00453f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:26.987635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151d7bb-f767-4a8d-b2fc-5753bf00453f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:26.994835 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 11/21] 10.222.0.36 () {56 vars in 1322 bytes} [Thu Aug 18 14:37:26 2022] GET /identity/v3/role_assignments?user.id=5064296cd28f403599a02675cb0c7103&scope.domain.id=default&role.id=9a439cf8eef34ada85428b378878e63e => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:28.990994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b21f46-15b8-4696-b80b-307f7674cfd0 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:28.991499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b21f46-15b8-4696-b80b-307f7674cfd0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:28.991603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b21f46-15b8-4696-b80b-307f7674cfd0 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:28.992867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 11/22] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:28.998833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284d4c6-0387-4e29-98fe-d1b582040cb4 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:28.999253 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284d4c6-0387-4e29-98fe-d1b582040cb4 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:28.999437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284d4c6-0387-4e29-98fe-d1b582040cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:29.028671 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-0284d4c6-0387-4e29-98fe-d1b582040cb4 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:29.029877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0284d4c6-0387-4e29-98fe-d1b582040cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:29.078488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e470c8-a960-4c2d-b102-13e7a719f734 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:29.079455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e470c8-a960-4c2d-b102-13e7a719f734 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:29.080501 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e470c8-a960-4c2d-b102-13e7a719f734 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:29.082353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 12/23] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:37:29.100194 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f03d1-621d-4493-ad1e-3ecd3129ca5c None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:29.101176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f03d1-621d-4493-ad1e-3ecd3129ca5c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:29.101931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f03d1-621d-4493-ad1e-3ecd3129ca5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:29.117087 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 12/24] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:37:29.131758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-ea1f03d1-621d-4493-ad1e-3ecd3129ca5c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:29.133265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ea1f03d1-621d-4493-ad1e-3ecd3129ca5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:29.219204 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 13/25] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:37:29.336837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299d21a-3204-4706-9a64-97323e895865 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:29.337117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299d21a-3204-4706-9a64-97323e895865 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:29.337392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299d21a-3204-4706-9a64-97323e895865 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:29.338987 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 13/26] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:29.343828 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4e758-f8c0-4c0b-834e-f6f64685bfe4 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:29.344115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4e758-f8c0-4c0b-834e-f6f64685bfe4 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:29.344389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4e758-f8c0-4c0b-834e-f6f64685bfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:29.368107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-29e4e758-f8c0-4c0b-834e-f6f64685bfe4 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:29.369291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-29e4e758-f8c0-4c0b-834e-f6f64685bfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:29.456320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 14/27] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:29.460811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299d21a-3204-4706-9a64-97323e895865 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:29.495178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:29.611472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299d21a-3204-4706-9a64-97323e895865 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:29.654042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeac74a-361d-40cf-959e-07b295e9da4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:29.654669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeac74a-361d-40cf-959e-07b295e9da4c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:29.654946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeac74a-361d-40cf-959e-07b295e9da4c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:29.655214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeac74a-361d-40cf-959e-07b295e9da4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:29.670802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeac74a-361d-40cf-959e-07b295e9da4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-bbeac74a-361d-40cf-959e-07b295e9da4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:29.677379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:29.678198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:37:29.678954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 14/28] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:37:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:37:29.686245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4e758-f8c0-4c0b-834e-f6f64685bfe4 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:29.716938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeac74a-361d-40cf-959e-07b295e9da4c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:29.728170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:29.860142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:29.878582 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeac74a-361d-40cf-959e-07b295e9da4c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:29.889635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4e758-f8c0-4c0b-834e-f6f64685bfe4 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:29.930623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce557b10-2938-4dc0-b2de-e6454aa55301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:29.931524 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce557b10-2938-4dc0-b2de-e6454aa55301 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:29.931963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce557b10-2938-4dc0-b2de-e6454aa55301 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:29.932374 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce557b10-2938-4dc0-b2de-e6454aa55301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:29.949291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce557b10-2938-4dc0-b2de-e6454aa55301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:29.951786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce557b10-2938-4dc0-b2de-e6454aa55301 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:29.969013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a698d-dc2c-4dfa-b907-3e80a21c9bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:29.970486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a698d-dc2c-4dfa-b907-3e80a21c9bed None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:29.972223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a698d-dc2c-4dfa-b907-3e80a21c9bed None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:29.972640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a698d-dc2c-4dfa-b907-3e80a21c9bed None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:29.974416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a698d-dc2c-4dfa-b907-3e80a21c9bed None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-ce557b10-2938-4dc0-b2de-e6454aa55301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:37:29.979786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:29.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:37:29.981841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:37:29.981841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:37:29.981841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:37:29.982410 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a698d-dc2c-4dfa-b907-3e80a21c9bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:29.984062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a698d-dc2c-4dfa-b907-3e80a21c9bed None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:29.986807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 15/29] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:37:29.988697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce557b10-2938-4dc0-b2de-e6454aa55301 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:29.997506 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 15/30] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:29.999571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a698d-dc2c-4dfa-b907-3e80a21c9bed None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:30.103354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:30.113647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce557b10-2938-4dc0-b2de-e6454aa55301 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:30.120686 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a698d-dc2c-4dfa-b907-3e80a21c9bed None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:30.153254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a9490-2710-49da-88ff-10d045afd947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:30.154026 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a9490-2710-49da-88ff-10d045afd947 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:30.154309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a9490-2710-49da-88ff-10d045afd947 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:30.154583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a9490-2710-49da-88ff-10d045afd947 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:30.155375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a9490-2710-49da-88ff-10d045afd947 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:30.156849 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a9490-2710-49da-88ff-10d045afd947 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:30.164747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2e4fc-61c5-4c0d-b5a0-2a88d8a79cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:30.165810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2e4fc-61c5-4c0d-b5a0-2a88d8a79cd6 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:30.166135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2e4fc-61c5-4c0d-b5a0-2a88d8a79cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:30.166461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2e4fc-61c5-4c0d-b5a0-2a88d8a79cd6 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:30.167164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2e4fc-61c5-4c0d-b5a0-2a88d8a79cd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:30.167536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2e4fc-61c5-4c0d-b5a0-2a88d8a79cd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:30.169218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2e4fc-61c5-4c0d-b5a0-2a88d8a79cd6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:30.175860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:30.195339 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 16/31] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:30.196228 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2e4fc-61c5-4c0d-b5a0-2a88d8a79cd6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:30.301275 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2e4fc-61c5-4c0d-b5a0-2a88d8a79cd6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:30.302033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 16/32] 10.222.0.36 () {60 vars in 1131 bytes} [Thu Aug 18 14:37:29 2022] POST /identity/v3/roles => generated 219 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:37:30.304112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b96e1-337c-4568-99ad-62b8a0ac923b None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:30.304803 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b96e1-337c-4568-99ad-62b8a0ac923b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:30.305477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b96e1-337c-4568-99ad-62b8a0ac923b None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:30.307991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 17/33] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:30.309138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b96e1-337c-4568-99ad-62b8a0ac923b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:30.377780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34b010-e06d-422e-8530-764550c2620e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:30.378663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34b010-e06d-422e-8530-764550c2620e None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:30.378768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34b010-e06d-422e-8530-764550c2620e None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:30.378996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34b010-e06d-422e-8530-764550c2620e None admin] PATH_INFO: `/v3/users/5064296cd28f403599a02675cb0c7103` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:30.380335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34b010-e06d-422e-8530-764550c2620e None admin] RBAC: Authorizing `identity:get_user(user_id=5064296cd28f403599a02675cb0c7103)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:30.381905 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34b010-e06d-422e-8530-764550c2620e None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:30.383664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 17/34] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:30 2022] GET /identity/v3/users/5064296cd28f403599a02675cb0c7103 => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:30.384982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad5da9-bb10-46f8-a4ed-de0ed46e1a88 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:30.385120 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad5da9-bb10-46f8-a4ed-de0ed46e1a88 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:30.385370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad5da9-bb10-46f8-a4ed-de0ed46e1a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:30.417962 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-57ad5da9-bb10-46f8-a4ed-de0ed46e1a88 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:30.419208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-57ad5da9-bb10-46f8-a4ed-de0ed46e1a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:30.429458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:30.445477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b96e1-337c-4568-99ad-62b8a0ac923b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:30.485309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c54b2-cf4f-4c28-af2c-751a91327c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:30.485986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c54b2-cf4f-4c28-af2c-751a91327c43 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:30.486842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c54b2-cf4f-4c28-af2c-751a91327c43 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:30.487174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c54b2-cf4f-4c28-af2c-751a91327c43 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:30.487850 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c54b2-cf4f-4c28-af2c-751a91327c43 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:30.489635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c54b2-cf4f-4c28-af2c-751a91327c43 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:30.516561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 18/35] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:30.518487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad5da9-bb10-46f8-a4ed-de0ed46e1a88 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:30.525554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 18/36] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:37:30 2022] POST /identity/v3/projects => generated 301 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:37:30.526619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72794-ac8f-4a97-8ccd-a4348dbe8759 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:30.526958 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72794-ac8f-4a97-8ccd-a4348dbe8759 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:30.527237 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72794-ac8f-4a97-8ccd-a4348dbe8759 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:30.528663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 19/37] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:30.537258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cd78c-d546-4aa6-9886-7b87c56f2bcb None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:30.537258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cd78c-d546-4aa6-9886-7b87c56f2bcb None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:30.537258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cd78c-d546-4aa6-9886-7b87c56f2bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:30.570080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3d8cd78c-d546-4aa6-9886-7b87c56f2bcb None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:30.573485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3d8cd78c-d546-4aa6-9886-7b87c56f2bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:30.679443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:30.707724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad5da9-bb10-46f8-a4ed-de0ed46e1a88 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:30.709937 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 20/38] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:30.752637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c3629-64eb-43b4-911f-73fd9f9dd9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:30.753283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c3629-64eb-43b4-911f-73fd9f9dd9f4 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:30.753563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c3629-64eb-43b4-911f-73fd9f9dd9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:30.753893 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c3629-64eb-43b4-911f-73fd9f9dd9f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:30.755017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c3629-64eb-43b4-911f-73fd9f9dd9f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:30.756556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c3629-64eb-43b4-911f-73fd9f9dd9f4 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:30.758362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 19/39] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:30.764084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cd78c-d546-4aa6-9886-7b87c56f2bcb None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:30.802703 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40000a92-503b-4bc9-925a-ce28dcabe217 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:30.802995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40000a92-503b-4bc9-925a-ce28dcabe217 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:30.803252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40000a92-503b-4bc9-925a-ce28dcabe217 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:30.804641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 20/40] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:37:30.809548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe702b-94bd-4418-90ac-df04d0493467 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:30.809832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe702b-94bd-4418-90ac-df04d0493467 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:30.810102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe702b-94bd-4418-90ac-df04d0493467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:30.834535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-effe702b-94bd-4418-90ac-df04d0493467 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:30.835782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-effe702b-94bd-4418-90ac-df04d0493467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:30.841093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:30.891123 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cd78c-d546-4aa6-9886-7b87c56f2bcb None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:30.927816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 21/41] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:37:30.928829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe702b-94bd-4418-90ac-df04d0493467 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:30.936664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e57b1b-3c7e-4b7f-8ca0-2ebecce55a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:30.937358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57b1b-3c7e-4b7f-8ca0-2ebecce55a24 None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:30.937706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57b1b-3c7e-4b7f-8ca0-2ebecce55a24 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:30.938050 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57b1b-3c7e-4b7f-8ca0-2ebecce55a24 None admin] PATH_INFO: `/v3/domains/default/users/5064296cd28f403599a02675cb0c7103/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:30.940589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e57b1b-3c7e-4b7f-8ca0-2ebecce55a24 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5064296cd28f403599a02675cb0c7103, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:30.942117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e57b1b-3c7e-4b7f-8ca0-2ebecce55a24 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:30.958610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 21/42] 10.222.0.36 () {58 vars in 1361 bytes} [Thu Aug 18 14:37:30 2022] PUT /identity/v3/domains/default/users/5064296cd28f403599a02675cb0c7103/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:37:31.013454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e57b1b-3c7e-4b7f-8ca0-2ebecce55a24 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:31.047073 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe702b-94bd-4418-90ac-df04d0493467 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:31.069348 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:31.089187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7dc10-8d49-47e1-a098-aa678cb5555a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:31.089864 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7dc10-8d49-47e1-a098-aa678cb5555a None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:31.090146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7dc10-8d49-47e1-a098-aa678cb5555a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:31.090686 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7dc10-8d49-47e1-a098-aa678cb5555a None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:31.091358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7dc10-8d49-47e1-a098-aa678cb5555a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:31.093612 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7dc10-8d49-47e1-a098-aa678cb5555a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:31.110746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:31.146543 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e57b1b-3c7e-4b7f-8ca0-2ebecce55a24 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:31.184558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abd44e-6470-4193-b616-a599806bcbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:31.185162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abd44e-6470-4193-b616-a599806bcbc5 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:31.185439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abd44e-6470-4193-b616-a599806bcbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:31.185708 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abd44e-6470-4193-b616-a599806bcbc5 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:31.186369 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abd44e-6470-4193-b616-a599806bcbc5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:31.187825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abd44e-6470-4193-b616-a599806bcbc5 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:31.202891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 22/43] 10.222.0.36 () {60 vars in 1131 bytes} [Thu Aug 18 14:37:31 2022] POST /identity/v3/roles => generated 223 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:37:31.216145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abd44e-6470-4193-b616-a599806bcbc5 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:31.227854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 22/44] 10.222.0.36 () {60 vars in 1131 bytes} [Thu Aug 18 14:37:30 2022] POST /identity/v3/roles => generated 225 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:37:31.444749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abd44e-6470-4193-b616-a599806bcbc5 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:31.446291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:31.483538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169c796-c14f-4c56-b487-57f1ceeea6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:31.484221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169c796-c14f-4c56-b487-57f1ceeea6e4 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:31.484498 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169c796-c14f-4c56-b487-57f1ceeea6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:31.484775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169c796-c14f-4c56-b487-57f1ceeea6e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:31.486022 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169c796-c14f-4c56-b487-57f1ceeea6e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:31.487975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169c796-c14f-4c56-b487-57f1ceeea6e4 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:31.489548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 23/45] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:31.495291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7dc10-8d49-47e1-a098-aa678cb5555a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:31.596593 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7dc10-8d49-47e1-a098-aa678cb5555a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:31.630035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63bc09-1dfc-4615-a2dc-00fbe4a7d860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:31.630648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63bc09-1dfc-4615-a2dc-00fbe4a7d860 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:31.630929 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63bc09-1dfc-4615-a2dc-00fbe4a7d860 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:31.631204 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63bc09-1dfc-4615-a2dc-00fbe4a7d860 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:31.631882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63bc09-1dfc-4615-a2dc-00fbe4a7d860 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:31.633451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63bc09-1dfc-4615-a2dc-00fbe4a7d860 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:31.667272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 23/46] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:37:31 2022] POST /identity/v3/projects => generated 312 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:37:32.978125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2b422-1448-4cf3-b4ab-f63ec83adbd2 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:32.978623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2b422-1448-4cf3-b4ab-f63ec83adbd2 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:32.978705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2b422-1448-4cf3-b4ab-f63ec83adbd2 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:32.980038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 24/47] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:32.985238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09fc56-1935-4939-8d0f-a89c4a2bd161 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:32.985625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09fc56-1935-4939-8d0f-a89c4a2bd161 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:32.985797 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09fc56-1935-4939-8d0f-a89c4a2bd161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:33.005409 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-ad09fc56-1935-4939-8d0f-a89c4a2bd161 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:33.007030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ad09fc56-1935-4939-8d0f-a89c4a2bd161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:33.052170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:33.072491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 24/48] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:33.335449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2b422-1448-4cf3-b4ab-f63ec83adbd2 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:33.433885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2b422-1448-4cf3-b4ab-f63ec83adbd2 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:33.466983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a310f36-7880-45f0-85c2-7a1f5bbefa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:33.467607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310f36-7880-45f0-85c2-7a1f5bbefa47 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:33.467919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310f36-7880-45f0-85c2-7a1f5bbefa47 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:33.468196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310f36-7880-45f0-85c2-7a1f5bbefa47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:33.477983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a310f36-7880-45f0-85c2-7a1f5bbefa47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-2a310f36-7880-45f0-85c2-7a1f5bbefa47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:33.482686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:33.483659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:37:33.485030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 25/49] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:37:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:33.490080 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09fc56-1935-4939-8d0f-a89c4a2bd161 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:33.554235 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d94f7-5b66-4f75-9d03-4bc26af58081 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:33.554521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d94f7-5b66-4f75-9d03-4bc26af58081 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:33.554797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d94f7-5b66-4f75-9d03-4bc26af58081 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:33.556165 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 26/50] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:37:33.560879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54741d75-ddc5-42e6-a76e-b70e57733992 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:33.561157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54741d75-ddc5-42e6-a76e-b70e57733992 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:33.561425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54741d75-ddc5-42e6-a76e-b70e57733992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:33.578607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-54741d75-ddc5-42e6-a76e-b70e57733992 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:33.580359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-54741d75-ddc5-42e6-a76e-b70e57733992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:33.587328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09fc56-1935-4939-8d0f-a89c4a2bd161 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:33.608273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:33.621397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9607b1-756d-47ab-9753-6b9bb84c3fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:33.622001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9607b1-756d-47ab-9753-6b9bb84c3fe6 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:33.622280 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9607b1-756d-47ab-9753-6b9bb84c3fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:33.622811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9607b1-756d-47ab-9753-6b9bb84c3fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:33.623485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9607b1-756d-47ab-9753-6b9bb84c3fe6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:33.623824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9607b1-756d-47ab-9753-6b9bb84c3fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:33.625218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9607b1-756d-47ab-9753-6b9bb84c3fe6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:33.633355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 25/51] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:33.638786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9607b1-756d-47ab-9753-6b9bb84c3fe6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:33.652074 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 27/52] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:37:33.683757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:33.735307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9607b1-756d-47ab-9753-6b9bb84c3fe6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:33.872950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d45c4-56f4-453e-876e-e3fa2bfcee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:33.873577 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d45c4-56f4-453e-876e-e3fa2bfcee96 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:33.873860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d45c4-56f4-453e-876e-e3fa2bfcee96 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:33.874186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d45c4-56f4-453e-876e-e3fa2bfcee96 None admin] PATH_INFO: `/v3/users/5064296cd28f403599a02675cb0c7103` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:33.875319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d45c4-56f4-453e-876e-e3fa2bfcee96 None admin] RBAC: Authorizing `identity:get_user(user_id=5064296cd28f403599a02675cb0c7103)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:33.877317 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d45c4-56f4-453e-876e-e3fa2bfcee96 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:33.878888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 26/53] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:33 2022] GET /identity/v3/users/5064296cd28f403599a02675cb0c7103 => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:33.884469 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54741d75-ddc5-42e6-a76e-b70e57733992 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:33.920358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d45c4-56f4-453e-876e-e3fa2bfcee96 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:33.987595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54741d75-ddc5-42e6-a76e-b70e57733992 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:34.018340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d45c4-56f4-453e-876e-e3fa2bfcee96 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:34.024894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1b558-2a17-443b-83ef-7083e32a0712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:34.025519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1b558-2a17-443b-83ef-7083e32a0712 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:34.025798 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1b558-2a17-443b-83ef-7083e32a0712 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:34.026071 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1b558-2a17-443b-83ef-7083e32a0712 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:34.027172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1b558-2a17-443b-83ef-7083e32a0712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:34.028661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1b558-2a17-443b-83ef-7083e32a0712 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:34.030219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 28/54] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:34.030867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:34.036018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1b558-2a17-443b-83ef-7083e32a0712 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:34.052394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79febd5-bbf9-4f6e-bf5e-f363c93ca166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:34.053054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79febd5-bbf9-4f6e-bf5e-f363c93ca166 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:34.053340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79febd5-bbf9-4f6e-bf5e-f363c93ca166 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:34.053619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79febd5-bbf9-4f6e-bf5e-f363c93ca166 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:34.054743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79febd5-bbf9-4f6e-bf5e-f363c93ca166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:34.056246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79febd5-bbf9-4f6e-bf5e-f363c93ca166 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:34.057814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 27/55] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:34.063116 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79febd5-bbf9-4f6e-bf5e-f363c93ca166 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:34.063226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:34.135725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1b558-2a17-443b-83ef-7083e32a0712 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:34.161186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79febd5-bbf9-4f6e-bf5e-f363c93ca166 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:34.169187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386d25f-391d-4f14-9954-b0b79832c50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:34.169780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386d25f-391d-4f14-9954-b0b79832c50f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:34.170063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386d25f-391d-4f14-9954-b0b79832c50f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:34.170335 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386d25f-391d-4f14-9954-b0b79832c50f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:34.171055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386d25f-391d-4f14-9954-b0b79832c50f None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.domain.id=default, user.id=5064296cd28f403599a02675cb0c7103) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:34.171361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386d25f-391d-4f14-9954-b0b79832c50f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:34.172832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386d25f-391d-4f14-9954-b0b79832c50f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:34.178568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 29/56] 10.222.0.36 () {56 vars in 1322 bytes} [Thu Aug 18 14:37:34 2022] GET /identity/v3/role_assignments?user.id=5064296cd28f403599a02675cb0c7103&scope.domain.id=default&role.id=9a439cf8eef34ada85428b378878e63e => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:34.194817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949e7c3-a802-4c9b-9d2e-9c9822e1106b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:34.195454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949e7c3-a802-4c9b-9d2e-9c9822e1106b None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:34.195765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949e7c3-a802-4c9b-9d2e-9c9822e1106b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:34.196040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949e7c3-a802-4c9b-9d2e-9c9822e1106b None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:34.196719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949e7c3-a802-4c9b-9d2e-9c9822e1106b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:34.198220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949e7c3-a802-4c9b-9d2e-9c9822e1106b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:34.228344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 28/57] 10.222.0.36 () {60 vars in 1140 bytes} [Thu Aug 18 14:37:34 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:37:36.146943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450a8b-4391-4e07-8dea-5de645dd5715 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:36.147427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450a8b-4391-4e07-8dea-5de645dd5715 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:36.147427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450a8b-4391-4e07-8dea-5de645dd5715 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:36.148774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 30/58] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:36.153996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680bcb9-4c17-4347-92b2-059dfa451be9 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:36.154403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680bcb9-4c17-4347-92b2-059dfa451be9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:36.154554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680bcb9-4c17-4347-92b2-059dfa451be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:36.172368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-2680bcb9-4c17-4347-92b2-059dfa451be9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:36.174189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2680bcb9-4c17-4347-92b2-059dfa451be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:36.198473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:36.239510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 29/59] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:36.509870 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450a8b-4391-4e07-8dea-5de645dd5715 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:36.598848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:36.608165 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450a8b-4391-4e07-8dea-5de645dd5715 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:36.640932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27734bee-b2f0-473d-aa6a-44df0efca7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:36.641540 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27734bee-b2f0-473d-aa6a-44df0efca7c8 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:36.641820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27734bee-b2f0-473d-aa6a-44df0efca7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:36.642094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27734bee-b2f0-473d-aa6a-44df0efca7c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:36.643209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27734bee-b2f0-473d-aa6a-44df0efca7c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:36.644687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27734bee-b2f0-473d-aa6a-44df0efca7c8 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:36.646252 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 31/60] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:36.651663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680bcb9-4c17-4347-92b2-059dfa451be9 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:36.747425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680bcb9-4c17-4347-92b2-059dfa451be9 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:36.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53889b2b-4502-475a-b732-fe69758181ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:36.780230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53889b2b-4502-475a-b732-fe69758181ef None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:36.780511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53889b2b-4502-475a-b732-fe69758181ef None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:36.780785 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53889b2b-4502-475a-b732-fe69758181ef None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:36.781419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53889b2b-4502-475a-b732-fe69758181ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:36.782895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53889b2b-4502-475a-b732-fe69758181ef None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:36.808569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 30/61] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:37:36 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:37:39.794383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb7db5-9aad-49dd-98cc-384b59a3c416 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:39.794929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb7db5-9aad-49dd-98cc-384b59a3c416 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:39.794929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb7db5-9aad-49dd-98cc-384b59a3c416 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:39.796282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 32/62] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:39.804969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf870e-2195-4de3-8b3b-0a075655ce45 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:39.805779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf870e-2195-4de3-8b3b-0a075655ce45 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:39.806268 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf870e-2195-4de3-8b3b-0a075655ce45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:39.832689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-36cf870e-2195-4de3-8b3b-0a075655ce45 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:39.834269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-36cf870e-2195-4de3-8b3b-0a075655ce45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:39.849155 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:39.933228 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 31/63] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:39.944103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f065aee-3a64-4443-9c58-b23abd4afff0 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:39.944698 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f065aee-3a64-4443-9c58-b23abd4afff0 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:39.945186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f065aee-3a64-4443-9c58-b23abd4afff0 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:39.946741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 33/64] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:39.952254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23e20-c78e-4aab-a032-64541427bd27 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:39.952543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23e20-c78e-4aab-a032-64541427bd27 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:39.952959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23e20-c78e-4aab-a032-64541427bd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:39.979648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-96d23e20-c78e-4aab-a032-64541427bd27 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:39.980882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-96d23e20-c78e-4aab-a032-64541427bd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:40.066715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 32/65] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:40.102204 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c99ea5-e217-495f-9d72-9d5a7973db5b None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:40.102809 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c99ea5-e217-495f-9d72-9d5a7973db5b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:40.103286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c99ea5-e217-495f-9d72-9d5a7973db5b None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:40.104875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 34/66] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:40.110160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2ea5c-35b1-496e-ad81-e0b60e4b7410 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:40.110453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2ea5c-35b1-496e-ad81-e0b60e4b7410 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:40.110727 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2ea5c-35b1-496e-ad81-e0b60e4b7410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:40.134166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-67a2ea5c-35b1-496e-ad81-e0b60e4b7410 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:40.135355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-67a2ea5c-35b1-496e-ad81-e0b60e4b7410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:40.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 33/67] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:40.277185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c99ea5-e217-495f-9d72-9d5a7973db5b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:40.370587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2ea5c-35b1-496e-ad81-e0b60e4b7410 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:40.392047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:40.402332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c99ea5-e217-495f-9d72-9d5a7973db5b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:40.437769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd178288-e0a8-404b-b4a5-f8445025f9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:40.438402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd178288-e0a8-404b-b4a5-f8445025f9da None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:40.438678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd178288-e0a8-404b-b4a5-f8445025f9da None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:40.438951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd178288-e0a8-404b-b4a5-f8445025f9da None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:40.450036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd178288-e0a8-404b-b4a5-f8445025f9da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-cd178288-e0a8-404b-b4a5-f8445025f9da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:40.455701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:40.456619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 35/68] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:40.461779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c6c7d-62eb-4b23-9355-2d7cc3f5d664 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:40.464443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c6c7d-62eb-4b23-9355-2d7cc3f5d664 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:40.464443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c6c7d-62eb-4b23-9355-2d7cc3f5d664 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:40.464443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 36/69] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:40.464443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d8406-db68-4d71-8e60-eea7ab18b08f None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:40.464443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d8406-db68-4d71-8e60-eea7ab18b08f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:40.464443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d8406-db68-4d71-8e60-eea7ab18b08f None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:40.464443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 37/70] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:40.467414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31d388-7258-47b8-846d-fc6c6c819f77 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:40.467710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31d388-7258-47b8-846d-fc6c6c819f77 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:40.468266 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31d388-7258-47b8-846d-fc6c6c819f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:40.486668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8c31d388-7258-47b8-846d-fc6c6c819f77 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:40.487896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8c31d388-7258-47b8-846d-fc6c6c819f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:40.502815 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:40.503609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2ea5c-35b1-496e-ad81-e0b60e4b7410 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:40.538625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b735f-2b73-4c65-9280-edb9a1c0e2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:40.539253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b735f-2b73-4c65-9280-edb9a1c0e2e3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:40.539533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b735f-2b73-4c65-9280-edb9a1c0e2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:40.539837 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b735f-2b73-4c65-9280-edb9a1c0e2e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:40.550386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b735f-2b73-4c65-9280-edb9a1c0e2e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-438b735f-2b73-4c65-9280-edb9a1c0e2e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:40.555117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:40.555998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:37:40.556752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 34/71] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:37:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:40.557920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b735f-2b73-4c65-9280-edb9a1c0e2e3 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:40.566176 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 38/72] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:40.567974 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784f62f-5482-4695-b734-65036f4da554 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:40.568344 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784f62f-5482-4695-b734-65036f4da554 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:40.568703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784f62f-5482-4695-b734-65036f4da554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:40.586810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-2784f62f-5482-4695-b734-65036f4da554 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:40.588120 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2784f62f-5482-4695-b734-65036f4da554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:40.640253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:40.661510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 39/73] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:40.663066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784f62f-5482-4695-b734-65036f4da554 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:40.674183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b735f-2b73-4c65-9280-edb9a1c0e2e3 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:40.709791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee2ee0-b81d-48d5-b1f6-a0a58dc57e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:40.710402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee2ee0-b81d-48d5-b1f6-a0a58dc57e6e None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:40.710683 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee2ee0-b81d-48d5-b1f6-a0a58dc57e6e None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:40.710957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee2ee0-b81d-48d5-b1f6-a0a58dc57e6e None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:40.711619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee2ee0-b81d-48d5-b1f6-a0a58dc57e6e None admin] RBAC: Adding query filter params (name=member) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:40.712009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee2ee0-b81d-48d5-b1f6-a0a58dc57e6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:40.713407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee2ee0-b81d-48d5-b1f6-a0a58dc57e6e None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:40.721874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 35/74] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:37:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:40.727234 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee2ee0-b81d-48d5-b1f6-a0a58dc57e6e None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:40.777684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:40.802161 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784f62f-5482-4695-b734-65036f4da554 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:40.838468 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb798bb6-b64e-473e-b097-99dc4d1f2f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:40.839385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb798bb6-b64e-473e-b097-99dc4d1f2f7c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:40.840206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb798bb6-b64e-473e-b097-99dc4d1f2f7c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:40.840484 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb798bb6-b64e-473e-b097-99dc4d1f2f7c None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:40.841151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb798bb6-b64e-473e-b097-99dc4d1f2f7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:40.841458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb798bb6-b64e-473e-b097-99dc4d1f2f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:40.842842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb798bb6-b64e-473e-b097-99dc4d1f2f7c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:40.850080 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee2ee0-b81d-48d5-b1f6-a0a58dc57e6e None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:40.855568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 40/75] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:40.857533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb798bb6-b64e-473e-b097-99dc4d1f2f7c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:40.881301 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:40.892040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31efb69-0232-4ca9-8951-f34342244a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:40.893318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31efb69-0232-4ca9-8951-f34342244a4c None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:40.893709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31efb69-0232-4ca9-8951-f34342244a4c None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:40.894096 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31efb69-0232-4ca9-8951-f34342244a4c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:40.905769 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31efb69-0232-4ca9-8951-f34342244a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-c31efb69-0232-4ca9-8951-f34342244a4c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:40.911121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:40.912004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:37:40.912773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 36/76] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:37:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:40.913515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31efb69-0232-4ca9-8951-f34342244a4c None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:40.966862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb798bb6-b64e-473e-b097-99dc4d1f2f7c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:40.999975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fd146-86dd-4e17-927e-1c3d653936cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:41.000597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fd146-86dd-4e17-927e-1c3d653936cd None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:41.000874 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fd146-86dd-4e17-927e-1c3d653936cd None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:41.001144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fd146-86dd-4e17-927e-1c3d653936cd None admin] PATH_INFO: `/v3/users/a485edf4c68a4d0eaed5f79e91691cc5` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:41.012536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:41.015281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fd146-86dd-4e17-927e-1c3d653936cd None admin] RBAC: Authorizing `identity:get_user(user_id=a485edf4c68a4d0eaed5f79e91691cc5)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:41.015418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31efb69-0232-4ca9-8951-f34342244a4c None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:41.017167 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fd146-86dd-4e17-927e-1c3d653936cd None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:41.023829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 41/77] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:40 2022] GET /identity/v3/users/a485edf4c68a4d0eaed5f79e91691cc5 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:41.023829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fd146-86dd-4e17-927e-1c3d653936cd None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:41.050198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8524603-1799-47b4-9154-9ec997a77e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:41.050823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8524603-1799-47b4-9154-9ec997a77e21 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:41.051102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8524603-1799-47b4-9154-9ec997a77e21 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:41.051379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8524603-1799-47b4-9154-9ec997a77e21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:41.061064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8524603-1799-47b4-9154-9ec997a77e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-e8524603-1799-47b4-9154-9ec997a77e21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:41.066794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:41.067654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:37:41.068466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 37/78] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:41.072533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8524603-1799-47b4-9154-9ec997a77e21 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:41.119411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fd146-86dd-4e17-927e-1c3d653936cd None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:41.120058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:41.153018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d3cd5-b18b-4ed5-9b57-e4adc526e41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:41.153580 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d3cd5-b18b-4ed5-9b57-e4adc526e41a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:41.153863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d3cd5-b18b-4ed5-9b57-e4adc526e41a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:41.154134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d3cd5-b18b-4ed5-9b57-e4adc526e41a None admin] PATH_INFO: `/v3/users/5064296cd28f403599a02675cb0c7103` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:41.155273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d3cd5-b18b-4ed5-9b57-e4adc526e41a None admin] RBAC: Authorizing `identity:get_user(user_id=5064296cd28f403599a02675cb0c7103)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:41.156766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d3cd5-b18b-4ed5-9b57-e4adc526e41a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:41.158510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 42/79] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:41 2022] GET /identity/v3/users/5064296cd28f403599a02675cb0c7103 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:41.159098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d3cd5-b18b-4ed5-9b57-e4adc526e41a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:41.170209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8524603-1799-47b4-9154-9ec997a77e21 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:41.203324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117d14f-ca1e-49e5-aeb2-fed326cad183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:41.203973 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117d14f-ca1e-49e5-aeb2-fed326cad183 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:41.204255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117d14f-ca1e-49e5-aeb2-fed326cad183 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:41.204533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117d14f-ca1e-49e5-aeb2-fed326cad183 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:41.205203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117d14f-ca1e-49e5-aeb2-fed326cad183 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:41.205514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117d14f-ca1e-49e5-aeb2-fed326cad183 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:41.206907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117d14f-ca1e-49e5-aeb2-fed326cad183 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:41.214590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 38/80] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:37:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:41.215367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117d14f-ca1e-49e5-aeb2-fed326cad183 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:41.224717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:41.257853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d3cd5-b18b-4ed5-9b57-e4adc526e41a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:41.291292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ef823-15a5-4eb3-a9c9-515649c1fa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:41.291941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ef823-15a5-4eb3-a9c9-515649c1fa46 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:41.292223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ef823-15a5-4eb3-a9c9-515649c1fa46 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:41.292495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ef823-15a5-4eb3-a9c9-515649c1fa46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:41.293613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ef823-15a5-4eb3-a9c9-515649c1fa46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:41.295048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ef823-15a5-4eb3-a9c9-515649c1fa46 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:41.296645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 43/81] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:41.297402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ef823-15a5-4eb3-a9c9-515649c1fa46 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:41.314743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117d14f-ca1e-49e5-aeb2-fed326cad183 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:41.327630 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:41.348588 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b9d33-26f0-4ccd-8f03-87f380aa53b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:41.349186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b9d33-26f0-4ccd-8f03-87f380aa53b8 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:41.349464 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b9d33-26f0-4ccd-8f03-87f380aa53b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:41.349742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b9d33-26f0-4ccd-8f03-87f380aa53b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:41.396245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ef823-15a5-4eb3-a9c9-515649c1fa46 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:41.414534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b9d33-26f0-4ccd-8f03-87f380aa53b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:41.415979 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b9d33-26f0-4ccd-8f03-87f380aa53b8 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:41.430341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d3257-1e31-495f-9b61-9a19f1190d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:41.430940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d3257-1e31-495f-9b61-9a19f1190d8d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:41.431224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d3257-1e31-495f-9b61-9a19f1190d8d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:41.431500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d3257-1e31-495f-9b61-9a19f1190d8d None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:41.432199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d3257-1e31-495f-9b61-9a19f1190d8d None admin] RBAC: Adding query filter params (name=member) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:41.432501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d3257-1e31-495f-9b61-9a19f1190d8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:41.433874 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d3257-1e31-495f-9b61-9a19f1190d8d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:41.436142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:41.443375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 44/82] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:37:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:41.444072 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d3257-1e31-495f-9b61-9a19f1190d8d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:41.513562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 39/83] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:37:41.513562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b9d33-26f0-4ccd-8f03-87f380aa53b8 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:41.528860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:41.544183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d3257-1e31-495f-9b61-9a19f1190d8d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:41.576794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693e974-ac2d-4957-9b1b-0e7dcd05b1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:41.577393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693e974-ac2d-4957-9b1b-0e7dcd05b1df None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:41.577674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693e974-ac2d-4957-9b1b-0e7dcd05b1df None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:41.577952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693e974-ac2d-4957-9b1b-0e7dcd05b1df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:41.610887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b9d33-26f0-4ccd-8f03-87f380aa53b8 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:41.639942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:41.644256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693e974-ac2d-4957-9b1b-0e7dcd05b1df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:41.644799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4ee44-18e8-4266-acbf-ea30774aeca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:41.645427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4ee44-18e8-4266-acbf-ea30774aeca9 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:41.645711 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4ee44-18e8-4266-acbf-ea30774aeca9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:41.645952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693e974-ac2d-4957-9b1b-0e7dcd05b1df None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:41.646042 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4ee44-18e8-4266-acbf-ea30774aeca9 None admin] PATH_INFO: `/v3/users/a485edf4c68a4d0eaed5f79e91691cc5` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:41.647106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4ee44-18e8-4266-acbf-ea30774aeca9 None admin] RBAC: Authorizing `identity:get_user(user_id=a485edf4c68a4d0eaed5f79e91691cc5)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:41.648571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4ee44-18e8-4266-acbf-ea30774aeca9 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:41.650142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 40/84] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:41 2022] GET /identity/v3/users/a485edf4c68a4d0eaed5f79e91691cc5 => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:41.650885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4ee44-18e8-4266-acbf-ea30774aeca9 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:41.727474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:41.744688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 45/85] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:37:41.745542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693e974-ac2d-4957-9b1b-0e7dcd05b1df None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:41.746900 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4ee44-18e8-4266-acbf-ea30774aeca9 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:41.780919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae3ffc-9532-459a-b1f9-fd8c114cc37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:41.781525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae3ffc-9532-459a-b1f9-fd8c114cc37c None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:41.781807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae3ffc-9532-459a-b1f9-fd8c114cc37c None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:41.782084 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae3ffc-9532-459a-b1f9-fd8c114cc37c None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:41.782724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae3ffc-9532-459a-b1f9-fd8c114cc37c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:41.784222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae3ffc-9532-459a-b1f9-fd8c114cc37c None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:41.806731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 41/86] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:37:41 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:37:41.807524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae3ffc-9532-459a-b1f9-fd8c114cc37c None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:41.846646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:41.852086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693e974-ac2d-4957-9b1b-0e7dcd05b1df None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:41.892819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca499a3-1a7d-4fee-a427-d95760a08325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:41.893817 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca499a3-1a7d-4fee-a427-d95760a08325 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:41.894096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca499a3-1a7d-4fee-a427-d95760a08325 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:41.894372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca499a3-1a7d-4fee-a427-d95760a08325 None admin] PATH_INFO: `/v3/users/a485edf4c68a4d0eaed5f79e91691cc5` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:41.896109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca499a3-1a7d-4fee-a427-d95760a08325 None admin] RBAC: Authorizing `identity:get_user(user_id=a485edf4c68a4d0eaed5f79e91691cc5)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:41.897616 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca499a3-1a7d-4fee-a427-d95760a08325 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:41.904626 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 46/87] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:41 2022] GET /identity/v3/users/a485edf4c68a4d0eaed5f79e91691cc5 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:41.904626 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca499a3-1a7d-4fee-a427-d95760a08325 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:41.933984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae3ffc-9532-459a-b1f9-fd8c114cc37c None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:41.977439 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c275f62-9c40-4eb0-b022-4da47236fa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:41.978141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c275f62-9c40-4eb0-b022-4da47236fa78 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:41.978360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c275f62-9c40-4eb0-b022-4da47236fa78 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:41.978638 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c275f62-9c40-4eb0-b022-4da47236fa78 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:41.979822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c275f62-9c40-4eb0-b022-4da47236fa78 None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:41.981679 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c275f62-9c40-4eb0-b022-4da47236fa78 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:41.983089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 42/88] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:41 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:41.983710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c275f62-9c40-4eb0-b022-4da47236fa78 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:41.990530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:42.027369 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca499a3-1a7d-4fee-a427-d95760a08325 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:42.064877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50ae0c-954f-4467-acd6-e3811e2ee6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:42.065494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50ae0c-954f-4467-acd6-e3811e2ee6d8 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:42.065773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50ae0c-954f-4467-acd6-e3811e2ee6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:42.066049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50ae0c-954f-4467-acd6-e3811e2ee6d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:42.108270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c275f62-9c40-4eb0-b022-4da47236fa78 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:42.110187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:42.149813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713dd48-bba1-4781-821f-9ba5f4506cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:42.150454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713dd48-bba1-4781-821f-9ba5f4506cbb None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:42.150735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713dd48-bba1-4781-821f-9ba5f4506cbb None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:42.151009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713dd48-bba1-4781-821f-9ba5f4506cbb None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:42.152971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713dd48-bba1-4781-821f-9ba5f4506cbb None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:42.153845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50ae0c-954f-4467-acd6-e3811e2ee6d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:42.154445 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713dd48-bba1-4781-821f-9ba5f4506cbb None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:42.155602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50ae0c-954f-4467-acd6-e3811e2ee6d8 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:42.156347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 43/89] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:41 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:42.156928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713dd48-bba1-4781-821f-9ba5f4506cbb None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:42.235012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:42.261085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713dd48-bba1-4781-821f-9ba5f4506cbb None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:42.291787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 47/90] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:37:42.292781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50ae0c-954f-4467-acd6-e3811e2ee6d8 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:42.308978 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c6fa2-64ad-44a3-9e16-9dba254f41d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:42.310206 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c6fa2-64ad-44a3-9e16-9dba254f41d2 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:42.310941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c6fa2-64ad-44a3-9e16-9dba254f41d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:42.311425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c6fa2-64ad-44a3-9e16-9dba254f41d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:42.366730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:42.398185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c6fa2-64ad-44a3-9e16-9dba254f41d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:42.400013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c6fa2-64ad-44a3-9e16-9dba254f41d2 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:42.410570 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50ae0c-954f-4467-acd6-e3811e2ee6d8 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:42.449738 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e4933-45da-42e0-adad-cb61cccda030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:42.450703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e4933-45da-42e0-adad-cb61cccda030 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:42.451507 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e4933-45da-42e0-adad-cb61cccda030 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:42.452052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e4933-45da-42e0-adad-cb61cccda030 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:42.453054 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e4933-45da-42e0-adad-cb61cccda030 None admin] RBAC: Adding query filter params (role.id=603573d2738b49c389ad913208d2ced1, scope.project.id=c37e716487e9421694a14be7fa5c2d0c, user.id=a485edf4c68a4d0eaed5f79e91691cc5) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:42.453568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e4933-45da-42e0-adad-cb61cccda030 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:42.455386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e4933-45da-42e0-adad-cb61cccda030 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:42.462766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 48/91] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:42 2022] GET /identity/v3/role_assignments?user.id=a485edf4c68a4d0eaed5f79e91691cc5&scope.project.id=c37e716487e9421694a14be7fa5c2d0c&role.id=603573d2738b49c389ad913208d2ced1 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:42.464234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e4933-45da-42e0-adad-cb61cccda030 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:42.479112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:42.561337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 44/92] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:37:42.562137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c6fa2-64ad-44a3-9e16-9dba254f41d2 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:42.648349 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e4933-45da-42e0-adad-cb61cccda030 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:42.684262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:42.692866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319f52c-938d-43a0-ad8e-c6b50844b512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:42.693731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319f52c-938d-43a0-ad8e-c6b50844b512 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:42.694021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319f52c-938d-43a0-ad8e-c6b50844b512 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:42.694298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319f52c-938d-43a0-ad8e-c6b50844b512 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:42.695455 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319f52c-938d-43a0-ad8e-c6b50844b512 None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=c37e716487e9421694a14be7fa5c2d0c, user.id=5064296cd28f403599a02675cb0c7103) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:42.695817 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319f52c-938d-43a0-ad8e-c6b50844b512 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:42.697430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319f52c-938d-43a0-ad8e-c6b50844b512 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:42.704049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 49/93] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:42 2022] GET /identity/v3/role_assignments?user.id=5064296cd28f403599a02675cb0c7103&scope.project.id=c37e716487e9421694a14be7fa5c2d0c&role.id=9a439cf8eef34ada85428b378878e63e => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:42.705290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319f52c-938d-43a0-ad8e-c6b50844b512 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:42.790497 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c6fa2-64ad-44a3-9e16-9dba254f41d2 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:42.836957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2416a80-2cd1-4d3d-9ef4-24c6dd83b0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:42.837530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2416a80-2cd1-4d3d-9ef4-24c6dd83b0e1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:42.837810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2416a80-2cd1-4d3d-9ef4-24c6dd83b0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:42.838091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2416a80-2cd1-4d3d-9ef4-24c6dd83b0e1 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:42.840501 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2416a80-2cd1-4d3d-9ef4-24c6dd83b0e1 None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:42.842081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2416a80-2cd1-4d3d-9ef4-24c6dd83b0e1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:42.848208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 45/94] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:42 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:42.855797 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2416a80-2cd1-4d3d-9ef4-24c6dd83b0e1 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:42.858749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319f52c-938d-43a0-ad8e-c6b50844b512 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:42.906818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95785de1-71b1-41c0-98ac-3f0896b61826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:42.908428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95785de1-71b1-41c0-98ac-3f0896b61826 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:42.908939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95785de1-71b1-41c0-98ac-3f0896b61826 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:42.909420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95785de1-71b1-41c0-98ac-3f0896b61826 None admin] PATH_INFO: `/v3/projects/b247e15407e4415ab7ca3c594925d8c4` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:42.910820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95785de1-71b1-41c0-98ac-3f0896b61826 None admin] RBAC: Authorizing `identity:get_project(project_id=b247e15407e4415ab7ca3c594925d8c4)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:42.913205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95785de1-71b1-41c0-98ac-3f0896b61826 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:42.916178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 50/95] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:42 2022] GET /identity/v3/projects/b247e15407e4415ab7ca3c594925d8c4 => generated 312 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:42.926465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95785de1-71b1-41c0-98ac-3f0896b61826 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:42.937854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:43.005726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:43.052888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95785de1-71b1-41c0-98ac-3f0896b61826 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:43.058047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2416a80-2cd1-4d3d-9ef4-24c6dd83b0e1 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:43.128645 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cad1ab-f3bf-423b-a243-9815cac71df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:43.129188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cad1ab-f3bf-423b-a243-9815cac71df0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:43.129472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cad1ab-f3bf-423b-a243-9815cac71df0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:43.130178 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cad1ab-f3bf-423b-a243-9815cac71df0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:43.130931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cad1ab-f3bf-423b-a243-9815cac71df0 None admin] RBAC: Adding query filter params (role.id=9df5a7d7691c49b9a78f0db0a050b7fe, scope.project.id=c37e716487e9421694a14be7fa5c2d0c, user.id=a485edf4c68a4d0eaed5f79e91691cc5) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:43.131244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cad1ab-f3bf-423b-a243-9815cac71df0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:43.132726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cad1ab-f3bf-423b-a243-9815cac71df0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:43.140484 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba0538-ce87-4ba3-ad22-b66e92c84b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:43.141121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba0538-ce87-4ba3-ad22-b66e92c84b62 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:43.141405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba0538-ce87-4ba3-ad22-b66e92c84b62 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:43.141688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba0538-ce87-4ba3-ad22-b66e92c84b62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:43.142498 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba0538-ce87-4ba3-ad22-b66e92c84b62 None admin] RBAC: Adding query filter params (role.id=603573d2738b49c389ad913208d2ced1, scope.project.id=b247e15407e4415ab7ca3c594925d8c4, user.id=a485edf4c68a4d0eaed5f79e91691cc5) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:43.142814 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba0538-ce87-4ba3-ad22-b66e92c84b62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:43.144316 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba0538-ce87-4ba3-ad22-b66e92c84b62 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:43.161706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 51/96] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:42 2022] GET /identity/v3/role_assignments?user.id=a485edf4c68a4d0eaed5f79e91691cc5&scope.project.id=b247e15407e4415ab7ca3c594925d8c4&role.id=603573d2738b49c389ad913208d2ced1 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:43.163537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 46/97] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:42 2022] GET /identity/v3/role_assignments?user.id=a485edf4c68a4d0eaed5f79e91691cc5&scope.project.id=c37e716487e9421694a14be7fa5c2d0c&role.id=9df5a7d7691c49b9a78f0db0a050b7fe => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:44.537366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9431ea-202e-4a75-bc12-e05d558f6d4d None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:44.537879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9431ea-202e-4a75-bc12-e05d558f6d4d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:44.537958 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9431ea-202e-4a75-bc12-e05d558f6d4d None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:44.539394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 52/98] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:44.544826 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604b425-cb4c-4c8e-acdd-378e3feaae76 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:44.545240 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604b425-cb4c-4c8e-acdd-378e3feaae76 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:44.545404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604b425-cb4c-4c8e-acdd-378e3feaae76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:44.565540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-d604b425-cb4c-4c8e-acdd-378e3feaae76 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:44.566752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d604b425-cb4c-4c8e-acdd-378e3feaae76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:44.639863 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 47/99] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:44.673199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba72887-1ea5-4dfb-8884-74e4ec9cb0f6 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:44.674096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba72887-1ea5-4dfb-8884-74e4ec9cb0f6 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:44.674429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba72887-1ea5-4dfb-8884-74e4ec9cb0f6 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:44.675782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 53/100] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:44.680993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9fc0d-5351-4d1f-945b-7162644db37c None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:44.681278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9fc0d-5351-4d1f-945b-7162644db37c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:44.681550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9fc0d-5351-4d1f-945b-7162644db37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:44.699511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-b1e9fc0d-5351-4d1f-945b-7162644db37c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:44.705326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b1e9fc0d-5351-4d1f-945b-7162644db37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:44.740447 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:44.780537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 48/101] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:45.023061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba72887-1ea5-4dfb-8884-74e4ec9cb0f6 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:45.136150 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9fc0d-5351-4d1f-945b-7162644db37c None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:45.160450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba72887-1ea5-4dfb-8884-74e4ec9cb0f6 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:45.210673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:45.228415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ccd12-e5b3-4a7b-a07b-531e7197da7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:45.229084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ccd12-e5b3-4a7b-a07b-531e7197da7f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:45.229368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ccd12-e5b3-4a7b-a07b-531e7197da7f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:45.229643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ccd12-e5b3-4a7b-a07b-531e7197da7f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:45.253989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ccd12-e5b3-4a7b-a07b-531e7197da7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-3f4ccd12-e5b3-4a7b-a07b-531e7197da7f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:45.259224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:45.260096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:45.260944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:45.260944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:45.260944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:45.260944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:45.260944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:45.260944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:45.260944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:45.260944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:45.260944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:45.260944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:45.260944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:37:45.260944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:37:45.263257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 54/102] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:45.264748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e9e8-cfcf-4eaa-a0ca-fc31068e164e None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:45.265231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e9e8-cfcf-4eaa-a0ca-fc31068e164e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:45.265796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e9e8-cfcf-4eaa-a0ca-fc31068e164e None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:45.267254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 55/103] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:45.268674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5cdc1-a431-4064-9ce4-0cc3396c78c2 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:45.269251 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5cdc1-a431-4064-9ce4-0cc3396c78c2 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:45.269721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5cdc1-a431-4064-9ce4-0cc3396c78c2 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:45.271121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 56/104] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:45.276333 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5cdc1-a431-4064-9ce4-0cc3396c78c2 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:45.285499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9fc0d-5351-4d1f-945b-7162644db37c None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:45.319979 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715fdb9-b959-45ea-ab43-585aa2333810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:45.320663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715fdb9-b959-45ea-ab43-585aa2333810 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:45.320945 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715fdb9-b959-45ea-ab43-585aa2333810 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:45.321649 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715fdb9-b959-45ea-ab43-585aa2333810 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:45.322810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715fdb9-b959-45ea-ab43-585aa2333810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:45.324401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715fdb9-b959-45ea-ab43-585aa2333810 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:45.326077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 49/105] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:45.331111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756c93f-5bbf-4036-af2a-170c13707066 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:45.331111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756c93f-5bbf-4036-af2a-170c13707066 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:45.331111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756c93f-5bbf-4036-af2a-170c13707066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:45.351871 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-0756c93f-5bbf-4036-af2a-170c13707066 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:45.354003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-0756c93f-5bbf-4036-af2a-170c13707066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:45.387108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:45.404897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5cdc1-a431-4064-9ce4-0cc3396c78c2 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:45.441784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 50/106] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:45.443320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfd5a6-4ed1-4ee6-9de3-5b665a737d46 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:45.443803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfd5a6-4ed1-4ee6-9de3-5b665a737d46 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:45.444256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfd5a6-4ed1-4ee6-9de3-5b665a737d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:45.456384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b95877-c11b-41fb-9bd0-4093d07f4511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:45.457169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b95877-c11b-41fb-9bd0-4093d07f4511 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:45.457942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b95877-c11b-41fb-9bd0-4093d07f4511 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:45.458300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b95877-c11b-41fb-9bd0-4093d07f4511 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:45.458989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b95877-c11b-41fb-9bd0-4093d07f4511 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:45.459295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b95877-c11b-41fb-9bd0-4093d07f4511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:45.464914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b95877-c11b-41fb-9bd0-4093d07f4511 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:45.472277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-2dcfd5a6-4ed1-4ee6-9de3-5b665a737d46 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:45.474034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2dcfd5a6-4ed1-4ee6-9de3-5b665a737d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:45.474832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 57/107] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:37:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:45.475765 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b95877-c11b-41fb-9bd0-4093d07f4511 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:45.570345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:45.603810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 51/108] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:45.604971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfd5a6-4ed1-4ee6-9de3-5b665a737d46 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:45.659008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b95877-c11b-41fb-9bd0-4093d07f4511 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:45.725115 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12ff61-bd26-4e46-9905-89c25f661ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:45.725651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12ff61-bd26-4e46-9905-89c25f661ca2 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:45.725931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12ff61-bd26-4e46-9905-89c25f661ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:45.726202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12ff61-bd26-4e46-9905-89c25f661ca2 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:45.726849 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12ff61-bd26-4e46-9905-89c25f661ca2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:45.728457 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12ff61-bd26-4e46-9905-89c25f661ca2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:45.791981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 58/109] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:37:45 2022] POST /identity/v3/projects => generated 302 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:37:45.871527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:45.914800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12ff61-bd26-4e46-9905-89c25f661ca2 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:45.914980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfd5a6-4ed1-4ee6-9de3-5b665a737d46 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:46.031250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38db1a0-b86f-49da-b83c-0ff7d4f676e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:46.031979 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38db1a0-b86f-49da-b83c-0ff7d4f676e0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:46.032265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38db1a0-b86f-49da-b83c-0ff7d4f676e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:46.032545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38db1a0-b86f-49da-b83c-0ff7d4f676e0 None admin] PATH_INFO: `/v3/users/c9c4f75f67d84d6d866bed009c7bbc03` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:46.062396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38db1a0-b86f-49da-b83c-0ff7d4f676e0 None admin] RBAC: Authorizing `identity:get_user(user_id=c9c4f75f67d84d6d866bed009c7bbc03)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:46.066144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38db1a0-b86f-49da-b83c-0ff7d4f676e0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:46.067271 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 52/110] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:45 2022] GET /identity/v3/users/c9c4f75f67d84d6d866bed009c7bbc03 => generated 289 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:46.067424 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38db1a0-b86f-49da-b83c-0ff7d4f676e0 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:46.151669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:46.159486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12ff61-bd26-4e46-9905-89c25f661ca2 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:46.198465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38db1a0-b86f-49da-b83c-0ff7d4f676e0 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:46.230015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a5644-dbc2-45e7-b95e-89b5f0e08e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:46.230658 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a5644-dbc2-45e7-b95e-89b5f0e08e81 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:46.230940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a5644-dbc2-45e7-b95e-89b5f0e08e81 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:46.231216 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a5644-dbc2-45e7-b95e-89b5f0e08e81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:46.238383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5b9ac-84c5-4c19-9879-9dc5f8a1f1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:46.239437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5b9ac-84c5-4c19-9879-9dc5f8a1f1ef None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:46.239927 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5b9ac-84c5-4c19-9879-9dc5f8a1f1ef None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:46.240307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5b9ac-84c5-4c19-9879-9dc5f8a1f1ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:46.241786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a5644-dbc2-45e7-b95e-89b5f0e08e81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-ed2a5644-dbc2-45e7-b95e-89b5f0e08e81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:46.249945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:46.250819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 59/111] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:46.256463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a5644-dbc2-45e7-b95e-89b5f0e08e81 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:46.260608 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5b9ac-84c5-4c19-9879-9dc5f8a1f1ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-47c5b9ac-84c5-4c19-9879-9dc5f8a1f1ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:46.265388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:46.266238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 53/112] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:37:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:46.268520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a1580-d92a-4015-9e97-c723881ffc38 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:46.269226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a1580-d92a-4015-9e97-c723881ffc38 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:46.269226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a1580-d92a-4015-9e97-c723881ffc38 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:46.270308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 54/113] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:46.271250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d8132-f222-403c-84c5-09eee75e4435 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:46.271735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d8132-f222-403c-84c5-09eee75e4435 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:46.271932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d8132-f222-403c-84c5-09eee75e4435 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:46.273712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 55/114] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:46.274468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d8132-f222-403c-84c5-09eee75e4435 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:46.291750 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:46.399140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a5644-dbc2-45e7-b95e-89b5f0e08e81 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:46.409834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d8132-f222-403c-84c5-09eee75e4435 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:46.436092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afc55f-88d2-43f6-87c5-9fbb812fe9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:46.436702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afc55f-88d2-43f6-87c5-9fbb812fe9ed None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:46.436980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afc55f-88d2-43f6-87c5-9fbb812fe9ed None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:46.437249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afc55f-88d2-43f6-87c5-9fbb812fe9ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:46.446132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd5ea2-ece3-47a8-a3aa-e22ff448d103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:46.447735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd5ea2-ece3-47a8-a3aa-e22ff448d103 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:46.447735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd5ea2-ece3-47a8-a3aa-e22ff448d103 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:46.447735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd5ea2-ece3-47a8-a3aa-e22ff448d103 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:46.447993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd5ea2-ece3-47a8-a3aa-e22ff448d103 None admin] RBAC: Adding query filter params (name=member) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:46.448255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd5ea2-ece3-47a8-a3aa-e22ff448d103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:46.449638 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd5ea2-ece3-47a8-a3aa-e22ff448d103 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:46.460387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 56/115] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:37:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:46.465609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd5ea2-ece3-47a8-a3aa-e22ff448d103 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:46.467907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:46.518291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afc55f-88d2-43f6-87c5-9fbb812fe9ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:46.519783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afc55f-88d2-43f6-87c5-9fbb812fe9ed None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:46.538601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:46.574118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd5ea2-ece3-47a8-a3aa-e22ff448d103 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:46.614088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c10f4-064c-4cc7-8f85-97fbb9b51213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:46.614713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c10f4-064c-4cc7-8f85-97fbb9b51213 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:46.614962 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c10f4-064c-4cc7-8f85-97fbb9b51213 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:46.615232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c10f4-064c-4cc7-8f85-97fbb9b51213 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:46.615938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c10f4-064c-4cc7-8f85-97fbb9b51213 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:46.616249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c10f4-064c-4cc7-8f85-97fbb9b51213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:46.617635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c10f4-064c-4cc7-8f85-97fbb9b51213 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:46.623923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 60/116] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:37:46.625007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15aff3-91e9-40b0-9bac-fd6d25792c55 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:46.625289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15aff3-91e9-40b0-9bac-fd6d25792c55 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:46.625559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15aff3-91e9-40b0-9bac-fd6d25792c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:46.631017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 57/117] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:46.631017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba98da0-4fa4-4cdb-ad13-bd200b02a018 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:46.631017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba98da0-4fa4-4cdb-ad13-bd200b02a018 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:46.631017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba98da0-4fa4-4cdb-ad13-bd200b02a018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:46.646840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-bba98da0-4fa4-4cdb-ad13-bd200b02a018 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:46.647000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-ed15aff3-91e9-40b0-9bac-fd6d25792c55 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:46.648069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bba98da0-4fa4-4cdb-ad13-bd200b02a018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:46.648170 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ed15aff3-91e9-40b0-9bac-fd6d25792c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:46.710035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:46.717136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 58/118] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:46.717926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba98da0-4fa4-4cdb-ad13-bd200b02a018 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:46.732219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 61/119] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:46.732219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15aff3-91e9-40b0-9bac-fd6d25792c55 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:46.841856 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:46.917883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15aff3-91e9-40b0-9bac-fd6d25792c55 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:46.928822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba98da0-4fa4-4cdb-ad13-bd200b02a018 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:46.985117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4df21e-f195-48f0-8353-9fa3b5e9f925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:46.986290 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df21e-f195-48f0-8353-9fa3b5e9f925 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:46.986570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df21e-f195-48f0-8353-9fa3b5e9f925 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:46.986842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df21e-f195-48f0-8353-9fa3b5e9f925 None admin] PATH_INFO: `/v3/users/a485edf4c68a4d0eaed5f79e91691cc5` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:46.988418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4df21e-f195-48f0-8353-9fa3b5e9f925 None admin] RBAC: Authorizing `identity:get_user(user_id=a485edf4c68a4d0eaed5f79e91691cc5)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:46.990682 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4df21e-f195-48f0-8353-9fa3b5e9f925 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:46.992573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 59/120] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:46 2022] GET /identity/v3/users/a485edf4c68a4d0eaed5f79e91691cc5 => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:46.993231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28284770-0dd1-4ff1-ad48-8f4b2e5a99a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:46.993370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4df21e-f195-48f0-8353-9fa3b5e9f925 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:46.994233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28284770-0dd1-4ff1-ad48-8f4b2e5a99a3 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:46.994717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28284770-0dd1-4ff1-ad48-8f4b2e5a99a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:46.995196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28284770-0dd1-4ff1-ad48-8f4b2e5a99a3 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:46.996602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28284770-0dd1-4ff1-ad48-8f4b2e5a99a3 None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:46.998587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28284770-0dd1-4ff1-ad48-8f4b2e5a99a3 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:47.000475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 62/121] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:46 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:47.001502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28284770-0dd1-4ff1-ad48-8f4b2e5a99a3 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:47.062247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:47.110039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28284770-0dd1-4ff1-ad48-8f4b2e5a99a3 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:47.112514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4df21e-f195-48f0-8353-9fa3b5e9f925 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:47.142848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:47.146254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8619ea-0b1c-4f02-b44b-c23779eb9ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:47.146884 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8619ea-0b1c-4f02-b44b-c23779eb9ab1 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:47.147164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8619ea-0b1c-4f02-b44b-c23779eb9ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:47.147436 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8619ea-0b1c-4f02-b44b-c23779eb9ab1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:47.149073 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3578ad9-6e0c-4399-b16d-3201d8f4ff4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:47.149903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3578ad9-6e0c-4399-b16d-3201d8f4ff4b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:47.150326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3578ad9-6e0c-4399-b16d-3201d8f4ff4b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:47.150743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3578ad9-6e0c-4399-b16d-3201d8f4ff4b None admin] PATH_INFO: `/v3/users/5064296cd28f403599a02675cb0c7103` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:47.152329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3578ad9-6e0c-4399-b16d-3201d8f4ff4b None admin] RBAC: Authorizing `identity:get_user(user_id=5064296cd28f403599a02675cb0c7103)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:47.154140 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3578ad9-6e0c-4399-b16d-3201d8f4ff4b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:47.155892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 60/122] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:46 2022] GET /identity/v3/users/5064296cd28f403599a02675cb0c7103 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:47.156800 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3578ad9-6e0c-4399-b16d-3201d8f4ff4b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:47.163505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8619ea-0b1c-4f02-b44b-c23779eb9ab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-ee8619ea-0b1c-4f02-b44b-c23779eb9ab1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:47.168281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:47.169134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:37:47.169904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 63/123] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:47.170527 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8619ea-0b1c-4f02-b44b-c23779eb9ab1 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:47.268788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3578ad9-6e0c-4399-b16d-3201d8f4ff4b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:47.276408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8619ea-0b1c-4f02-b44b-c23779eb9ab1 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:47.292906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:47.306289 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f588ce5-2de1-4f48-bb32-0060447e7cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:47.307142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f588ce5-2de1-4f48-bb32-0060447e7cfe None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:47.307601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f588ce5-2de1-4f48-bb32-0060447e7cfe None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:47.308102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f588ce5-2de1-4f48-bb32-0060447e7cfe None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:47.309404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f588ce5-2de1-4f48-bb32-0060447e7cfe None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:47.310548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9283e5-1cc2-4301-b328-504f1bfee737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:47.311145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9283e5-1cc2-4301-b328-504f1bfee737 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:47.311425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9283e5-1cc2-4301-b328-504f1bfee737 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:47.311639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f588ce5-2de1-4f48-bb32-0060447e7cfe None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:47.311765 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9283e5-1cc2-4301-b328-504f1bfee737 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:47.312441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9283e5-1cc2-4301-b328-504f1bfee737 None admin] RBAC: Adding query filter params (role.id=a853ef2aaba2459f86b50cae95d23577, scope.project.id=c37e716487e9421694a14be7fa5c2d0c, user.id=c9c4f75f67d84d6d866bed009c7bbc03) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:47.312746 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9283e5-1cc2-4301-b328-504f1bfee737 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:47.314030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 61/124] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:47 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:47.314238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9283e5-1cc2-4301-b328-504f1bfee737 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:47.315131 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f588ce5-2de1-4f48-bb32-0060447e7cfe None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:47.323811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 64/125] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:47 2022] GET /identity/v3/role_assignments?user.id=c9c4f75f67d84d6d866bed009c7bbc03&scope.project.id=c37e716487e9421694a14be7fa5c2d0c&role.id=a853ef2aaba2459f86b50cae95d23577 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:47.324635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9283e5-1cc2-4301-b328-504f1bfee737 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:47.393331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:47.432233 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f588ce5-2de1-4f48-bb32-0060447e7cfe None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:47.457289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9283e5-1cc2-4301-b328-504f1bfee737 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:47.471235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6c4f6-7ba2-4ce7-8676-aebaa20a3f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:47.472257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6c4f6-7ba2-4ce7-8676-aebaa20a3f19 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:47.472738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6c4f6-7ba2-4ce7-8676-aebaa20a3f19 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:47.473282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6c4f6-7ba2-4ce7-8676-aebaa20a3f19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:47.486439 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6c4f6-7ba2-4ce7-8676-aebaa20a3f19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-bda6c4f6-7ba2-4ce7-8676-aebaa20a3f19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:47.492128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:47.492999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:47.493770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:47.493770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:47.493770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:47.493770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:47.493770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:47.493770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:47.493770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:47.493770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:47.493770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:47.493770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:47.493770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:37:47.493770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:37:47.498443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 62/126] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:37:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:47.499371 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6c4f6-7ba2-4ce7-8676-aebaa20a3f19 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:47.506734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acdcf2-f179-4536-af1c-68bf147e047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:47.507632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acdcf2-f179-4536-af1c-68bf147e047d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:47.508137 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acdcf2-f179-4536-af1c-68bf147e047d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:47.508600 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acdcf2-f179-4536-af1c-68bf147e047d None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:47.509962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acdcf2-f179-4536-af1c-68bf147e047d None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:47.511609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acdcf2-f179-4536-af1c-68bf147e047d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:47.513544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 65/127] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:47 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:47.514687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acdcf2-f179-4536-af1c-68bf147e047d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:47.576146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:47.620582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6c4f6-7ba2-4ce7-8676-aebaa20a3f19 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:47.631920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acdcf2-f179-4536-af1c-68bf147e047d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:47.651508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:47.660931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da12028-6410-42c1-8b09-d151efcc822f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:47.661734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da12028-6410-42c1-8b09-d151efcc822f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:47.662154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da12028-6410-42c1-8b09-d151efcc822f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:47.662527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da12028-6410-42c1-8b09-d151efcc822f None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:47.663367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da12028-6410-42c1-8b09-d151efcc822f None admin] RBAC: Adding query filter params (name=member) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:47.664053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da12028-6410-42c1-8b09-d151efcc822f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:47.665318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da12028-6410-42c1-8b09-d151efcc822f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:47.673929 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 63/128] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:37:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:47.676738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da12028-6410-42c1-8b09-d151efcc822f None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:47.682827 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a55f4-2d8f-42a1-88f3-295f365bbae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:47.683516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a55f4-2d8f-42a1-88f3-295f365bbae5 None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:47.683830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a55f4-2d8f-42a1-88f3-295f365bbae5 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:47.684110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a55f4-2d8f-42a1-88f3-295f365bbae5 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c/users/a485edf4c68a4d0eaed5f79e91691cc5/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:47.689616 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a55f4-2d8f-42a1-88f3-295f365bbae5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c37e716487e9421694a14be7fa5c2d0c, user_id=a485edf4c68a4d0eaed5f79e91691cc5, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:47.691488 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a55f4-2d8f-42a1-88f3-295f365bbae5 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:47.708147 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 66/129] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:37:47 2022] PUT /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c/users/a485edf4c68a4d0eaed5f79e91691cc5/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:37:47.709328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a55f4-2d8f-42a1-88f3-295f365bbae5 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:47.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a55f4-2d8f-42a1-88f3-295f365bbae5 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:47.877584 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da12028-6410-42c1-8b09-d151efcc822f None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:47.889965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:47.923937 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf45d1-428a-4345-90e5-2b96a9379d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:47.924952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf45d1-428a-4345-90e5-2b96a9379d00 None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:47.925433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf45d1-428a-4345-90e5-2b96a9379d00 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:47.925907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf45d1-428a-4345-90e5-2b96a9379d00 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c/users/5064296cd28f403599a02675cb0c7103/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:47.929584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf45d1-428a-4345-90e5-2b96a9379d00 None admin] RBAC: Authorizing `identity:create_grant(project_id=c37e716487e9421694a14be7fa5c2d0c, user_id=5064296cd28f403599a02675cb0c7103, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:47.931290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf45d1-428a-4345-90e5-2b96a9379d00 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:47.932330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2368a9a-fae1-47c7-ad2e-7a67c9c42cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:47.932954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2368a9a-fae1-47c7-ad2e-7a67c9c42cb6 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:47.933241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2368a9a-fae1-47c7-ad2e-7a67c9c42cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:47.933519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2368a9a-fae1-47c7-ad2e-7a67c9c42cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:47.934196 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2368a9a-fae1-47c7-ad2e-7a67c9c42cb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:47.934503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2368a9a-fae1-47c7-ad2e-7a67c9c42cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:47.937667 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2368a9a-fae1-47c7-ad2e-7a67c9c42cb6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:47.950841 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 64/130] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:37:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:47.951572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2368a9a-fae1-47c7-ad2e-7a67c9c42cb6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:47.960196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 67/131] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:37:47 2022] PUT /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c/users/5064296cd28f403599a02675cb0c7103/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:37:47.961823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb777ef9-8d9b-4f82-8ab7-3e8afbbfc67b None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:47.962188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb777ef9-8d9b-4f82-8ab7-3e8afbbfc67b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:47.962642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb777ef9-8d9b-4f82-8ab7-3e8afbbfc67b None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:47.964397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 68/132] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:47.965367 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb777ef9-8d9b-4f82-8ab7-3e8afbbfc67b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:48.025166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:48.113313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2368a9a-fae1-47c7-ad2e-7a67c9c42cb6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:48.152844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb777ef9-8d9b-4f82-8ab7-3e8afbbfc67b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:48.161026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1140f88-f863-4f39-9398-2fdeeca670d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:48.161728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1140f88-f863-4f39-9398-2fdeeca670d3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:48.162021 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1140f88-f863-4f39-9398-2fdeeca670d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:48.162787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1140f88-f863-4f39-9398-2fdeeca670d3 None admin] PATH_INFO: `/v3/users/a485edf4c68a4d0eaed5f79e91691cc5` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:48.165147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1140f88-f863-4f39-9398-2fdeeca670d3 None admin] RBAC: Authorizing `identity:get_user(user_id=a485edf4c68a4d0eaed5f79e91691cc5)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:48.167255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1140f88-f863-4f39-9398-2fdeeca670d3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:48.169772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 65/133] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:47 2022] GET /identity/v3/users/a485edf4c68a4d0eaed5f79e91691cc5 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:48.176368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad250065-4f66-4e34-992d-c7023ca47859 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:48.176661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad250065-4f66-4e34-992d-c7023ca47859 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:48.176943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad250065-4f66-4e34-992d-c7023ca47859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:48.198320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7476d-ccc8-4274-8f7c-aeb5f2debf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:48.199189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7476d-ccc8-4274-8f7c-aeb5f2debf46 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:48.199505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-ad250065-4f66-4e34-992d-c7023ca47859 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:48.199797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7476d-ccc8-4274-8f7c-aeb5f2debf46 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:48.200207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7476d-ccc8-4274-8f7c-aeb5f2debf46 None admin] PATH_INFO: `/v3/users/a485edf4c68a4d0eaed5f79e91691cc5` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:48.201127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ad250065-4f66-4e34-992d-c7023ca47859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:48.201653 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7476d-ccc8-4274-8f7c-aeb5f2debf46 None admin] RBAC: Authorizing `identity:get_user(user_id=a485edf4c68a4d0eaed5f79e91691cc5)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:48.203508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7476d-ccc8-4274-8f7c-aeb5f2debf46 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:48.206290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 69/134] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:47 2022] GET /identity/v3/users/a485edf4c68a4d0eaed5f79e91691cc5 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:48.207212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7476d-ccc8-4274-8f7c-aeb5f2debf46 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:48.266212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:48.290685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 66/135] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:48.291790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad250065-4f66-4e34-992d-c7023ca47859 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:48.390676 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7476d-ccc8-4274-8f7c-aeb5f2debf46 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:48.401888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:48.435899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cd094-5ec4-4d75-8cfe-11b6e7181544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:48.435899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cd094-5ec4-4d75-8cfe-11b6e7181544 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:48.436175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cd094-5ec4-4d75-8cfe-11b6e7181544 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:48.436529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cd094-5ec4-4d75-8cfe-11b6e7181544 None admin] PATH_INFO: `/v3/projects/b247e15407e4415ab7ca3c594925d8c4` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:48.437791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cd094-5ec4-4d75-8cfe-11b6e7181544 None admin] RBAC: Authorizing `identity:get_project(project_id=b247e15407e4415ab7ca3c594925d8c4)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:48.439979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cd094-5ec4-4d75-8cfe-11b6e7181544 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:48.442208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 70/136] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:48 2022] GET /identity/v3/projects/b247e15407e4415ab7ca3c594925d8c4 => generated 312 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:48.451719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cd094-5ec4-4d75-8cfe-11b6e7181544 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:48.472940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad250065-4f66-4e34-992d-c7023ca47859 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:48.524522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48500b5-2fb7-47e5-aa7c-4bd8c3fe2fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:48.525509 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48500b5-2fb7-47e5-aa7c-4bd8c3fe2fb9 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:48.525985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48500b5-2fb7-47e5-aa7c-4bd8c3fe2fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:48.526425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48500b5-2fb7-47e5-aa7c-4bd8c3fe2fb9 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:48.532399 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48500b5-2fb7-47e5-aa7c-4bd8c3fe2fb9 None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:48.534691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48500b5-2fb7-47e5-aa7c-4bd8c3fe2fb9 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:48.538926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 67/137] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:48 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:48.544485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48500b5-2fb7-47e5-aa7c-4bd8c3fe2fb9 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:48.701649 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48500b5-2fb7-47e5-aa7c-4bd8c3fe2fb9 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:48.705887 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cd094-5ec4-4d75-8cfe-11b6e7181544 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:48.719487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:48.736480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:48.760962 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c101f-8e56-43ab-84d6-d6ea19405b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:48.761624 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c101f-8e56-43ab-84d6-d6ea19405b75 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:48.761903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c101f-8e56-43ab-84d6-d6ea19405b75 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:48.762174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c101f-8e56-43ab-84d6-d6ea19405b75 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c/users/a485edf4c68a4d0eaed5f79e91691cc5/roles/9df5a7d7691c49b9a78f0db0a050b7fe` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:48.766458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c101f-8e56-43ab-84d6-d6ea19405b75 None admin] RBAC: Authorizing `identity:create_grant(project_id=c37e716487e9421694a14be7fa5c2d0c, user_id=a485edf4c68a4d0eaed5f79e91691cc5, role_id=9df5a7d7691c49b9a78f0db0a050b7fe)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:48.768470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c101f-8e56-43ab-84d6-d6ea19405b75 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:48.776623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dc145-ce2e-4f37-bbdc-6be70dc59c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:48.777281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dc145-ce2e-4f37-bbdc-6be70dc59c2e None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:48.777565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dc145-ce2e-4f37-bbdc-6be70dc59c2e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:48.777838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dc145-ce2e-4f37-bbdc-6be70dc59c2e None admin] PATH_INFO: `/v3/projects/b247e15407e4415ab7ca3c594925d8c4/users/a485edf4c68a4d0eaed5f79e91691cc5/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:48.780739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dc145-ce2e-4f37-bbdc-6be70dc59c2e None admin] RBAC: Authorizing `identity:create_grant(project_id=b247e15407e4415ab7ca3c594925d8c4, user_id=a485edf4c68a4d0eaed5f79e91691cc5, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:48.782234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dc145-ce2e-4f37-bbdc-6be70dc59c2e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:48.793881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 68/138] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:37:48 2022] PUT /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c/users/a485edf4c68a4d0eaed5f79e91691cc5/roles/9df5a7d7691c49b9a78f0db0a050b7fe => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:37:48.800158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c101f-8e56-43ab-84d6-d6ea19405b75 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:48.807382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 71/139] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:37:48 2022] PUT /identity/v3/projects/b247e15407e4415ab7ca3c594925d8c4/users/a485edf4c68a4d0eaed5f79e91691cc5/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:37:48.958434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c101f-8e56-43ab-84d6-d6ea19405b75 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:49.003542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc41dca-9e10-4a9d-8dca-442f66bae095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:49.004231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc41dca-9e10-4a9d-8dca-442f66bae095 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:49.004518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc41dca-9e10-4a9d-8dca-442f66bae095 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:49.004792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc41dca-9e10-4a9d-8dca-442f66bae095 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:49.005931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc41dca-9e10-4a9d-8dca-442f66bae095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:49.007400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc41dca-9e10-4a9d-8dca-442f66bae095 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:49.009003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 69/140] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:49.016821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dc145-ce2e-4f37-bbdc-6be70dc59c2e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:49.140024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dc145-ce2e-4f37-bbdc-6be70dc59c2e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:49.176744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a93f50-a968-4d1f-9cb7-6402e9c194bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:49.177383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a93f50-a968-4d1f-9cb7-6402e9c194bb None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:49.177635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a93f50-a968-4d1f-9cb7-6402e9c194bb None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:49.177908 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a93f50-a968-4d1f-9cb7-6402e9c194bb None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:49.178561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a93f50-a968-4d1f-9cb7-6402e9c194bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:49.180026 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a93f50-a968-4d1f-9cb7-6402e9c194bb None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:49.213042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 72/141] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:37:49 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:37:49.844695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d2120-0994-4c5c-8cdd-ad2a55aabd09 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:49.845731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d2120-0994-4c5c-8cdd-ad2a55aabd09 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:49.845830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d2120-0994-4c5c-8cdd-ad2a55aabd09 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:49.847266 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 70/142] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:49.852749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f22c09-f168-4c6b-8f8d-5c03ef7dc8c4 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:49.853679 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f22c09-f168-4c6b-8f8d-5c03ef7dc8c4 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:49.854150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f22c09-f168-4c6b-8f8d-5c03ef7dc8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:49.873837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-b2f22c09-f168-4c6b-8f8d-5c03ef7dc8c4 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:49.875343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b2f22c09-f168-4c6b-8f8d-5c03ef7dc8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:49.894000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:49.947809 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 73/143] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:37:50.327075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d2120-0994-4c5c-8cdd-ad2a55aabd09 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:50.390526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:50.438820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506e46a-51bb-4118-ba62-29031b51e321 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:50.439465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506e46a-51bb-4118-ba62-29031b51e321 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:50.439985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506e46a-51bb-4118-ba62-29031b51e321 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:50.441514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 74/144] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:50.446926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31845248-0865-4f73-99e6-382311acf921 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:50.447487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31845248-0865-4f73-99e6-382311acf921 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:50.447984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31845248-0865-4f73-99e6-382311acf921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:50.464386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d2120-0994-4c5c-8cdd-ad2a55aabd09 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:50.481451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-31845248-0865-4f73-99e6-382311acf921 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:50.482987 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-31845248-0865-4f73-99e6-382311acf921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:50.509387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38b0bc-83c0-48c4-b33c-ef0ad99dbf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:50.510053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38b0bc-83c0-48c4-b33c-ef0ad99dbf02 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:50.510334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38b0bc-83c0-48c4-b33c-ef0ad99dbf02 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:50.510606 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38b0bc-83c0-48c4-b33c-ef0ad99dbf02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:50.522361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38b0bc-83c0-48c4-b33c-ef0ad99dbf02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-1c38b0bc-83c0-48c4-b33c-ef0ad99dbf02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:50.527156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:50.528053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:37:50.528819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 71/145] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:50.535050 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38b0bc-83c0-48c4-b33c-ef0ad99dbf02 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:50.606117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 75/146] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:50.678598 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38b0bc-83c0-48c4-b33c-ef0ad99dbf02 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:50.686435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:50.716799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b738b-ea81-490e-9a1b-94f7c7de7708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:50.717410 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b738b-ea81-490e-9a1b-94f7c7de7708 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:50.717684 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b738b-ea81-490e-9a1b-94f7c7de7708 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:50.717953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b738b-ea81-490e-9a1b-94f7c7de7708 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:50.718620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b738b-ea81-490e-9a1b-94f7c7de7708 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:50.718917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b738b-ea81-490e-9a1b-94f7c7de7708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:50.720386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b738b-ea81-490e-9a1b-94f7c7de7708 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:50.729076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 72/147] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:37:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:50.734610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31845248-0865-4f73-99e6-382311acf921 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:50.792634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dd599-909f-4d0e-b157-aab74b24ffd6 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:50.792938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dd599-909f-4d0e-b157-aab74b24ffd6 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:50.793174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dd599-909f-4d0e-b157-aab74b24ffd6 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:50.794514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 73/148] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:37:50.808799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4477e5-bc68-4f11-845d-5475193a309d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:50.809075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4477e5-bc68-4f11-845d-5475193a309d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:50.809406 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4477e5-bc68-4f11-845d-5475193a309d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:50.839644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-8c4477e5-bc68-4f11-845d-5475193a309d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:50.840870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8c4477e5-bc68-4f11-845d-5475193a309d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:50.879013 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:50.919464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31845248-0865-4f73-99e6-382311acf921 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:50.969060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 74/149] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:37:50.984548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b06829-782b-425a-be16-c4353f5d79b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:50.985249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b06829-782b-425a-be16-c4353f5d79b4 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:50.985528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b06829-782b-425a-be16-c4353f5d79b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:50.986331 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b06829-782b-425a-be16-c4353f5d79b4 None admin] PATH_INFO: `/v3/users/c9c4f75f67d84d6d866bed009c7bbc03` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:50.987465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b06829-782b-425a-be16-c4353f5d79b4 None admin] RBAC: Authorizing `identity:get_user(user_id=c9c4f75f67d84d6d866bed009c7bbc03)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:50.989013 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b06829-782b-425a-be16-c4353f5d79b4 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:50.990876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 76/150] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:50 2022] GET /identity/v3/users/c9c4f75f67d84d6d866bed009c7bbc03 => generated 289 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:50.996673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4477e5-bc68-4f11-845d-5475193a309d None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:51.018056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b06829-782b-425a-be16-c4353f5d79b4 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:51.169251 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b06829-782b-425a-be16-c4353f5d79b4 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:51.183036 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:51.188316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4477e5-bc68-4f11-845d-5475193a309d None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:51.226871 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb258c41-4551-4267-a3b3-a49c7fb8fc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:51.227912 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb258c41-4551-4267-a3b3-a49c7fb8fc01 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:51.228356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828081c5-31b8-42cc-8c66-d153246d18f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:51.228618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb258c41-4551-4267-a3b3-a49c7fb8fc01 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:51.228989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828081c5-31b8-42cc-8c66-d153246d18f7 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:51.229266 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828081c5-31b8-42cc-8c66-d153246d18f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:51.229470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb258c41-4551-4267-a3b3-a49c7fb8fc01 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:51.229563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828081c5-31b8-42cc-8c66-d153246d18f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:51.230868 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb258c41-4551-4267-a3b3-a49c7fb8fc01 None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:51.232779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb258c41-4551-4267-a3b3-a49c7fb8fc01 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:51.234653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 75/151] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:50 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:51.241756 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb258c41-4551-4267-a3b3-a49c7fb8fc01 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:51.245004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828081c5-31b8-42cc-8c66-d153246d18f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-828081c5-31b8-42cc-8c66-d153246d18f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:51.250063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:51.251058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:51.252399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:51.252399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:51.252399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:51.252399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:51.252399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:51.252399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:51.252399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:51.252399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:51.252399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:51.252399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:51.252399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:51.252399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:37:51.255826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 77/152] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:37:51.259665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828081c5-31b8-42cc-8c66-d153246d18f7 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:51.276070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:51.362905 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb258c41-4551-4267-a3b3-a49c7fb8fc01 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:51.400336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37425382-f07c-43d2-828e-859fe2f77268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:51.401441 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37425382-f07c-43d2-828e-859fe2f77268 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:51.401992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37425382-f07c-43d2-828e-859fe2f77268 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:51.402454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37425382-f07c-43d2-828e-859fe2f77268 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c/users/c9c4f75f67d84d6d866bed009c7bbc03/roles/a853ef2aaba2459f86b50cae95d23577` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:51.403925 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828081c5-31b8-42cc-8c66-d153246d18f7 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:51.404788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37425382-f07c-43d2-828e-859fe2f77268 None admin] RBAC: Authorizing `identity:create_grant(project_id=c37e716487e9421694a14be7fa5c2d0c, user_id=c9c4f75f67d84d6d866bed009c7bbc03, role_id=a853ef2aaba2459f86b50cae95d23577)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:51.406640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37425382-f07c-43d2-828e-859fe2f77268 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:51.423755 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 76/153] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:37:51 2022] PUT /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c/users/c9c4f75f67d84d6d866bed009c7bbc03/roles/a853ef2aaba2459f86b50cae95d23577 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:37:51.425070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37425382-f07c-43d2-828e-859fe2f77268 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:51.461169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d199eea-2a5c-48eb-8071-2e75091c7454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:51.461789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d199eea-2a5c-48eb-8071-2e75091c7454 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:51.462062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d199eea-2a5c-48eb-8071-2e75091c7454 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:51.462332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d199eea-2a5c-48eb-8071-2e75091c7454 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:51.463012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d199eea-2a5c-48eb-8071-2e75091c7454 None admin] RBAC: Adding query filter params (name=member) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:51.463411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d199eea-2a5c-48eb-8071-2e75091c7454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:51.465878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d199eea-2a5c-48eb-8071-2e75091c7454 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:51.469548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:51.479673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 78/154] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:37:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:51.485392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d199eea-2a5c-48eb-8071-2e75091c7454 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:51.595476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:51.617166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37425382-f07c-43d2-828e-859fe2f77268 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:51.640890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d199eea-2a5c-48eb-8071-2e75091c7454 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:51.668406 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f787a1-73c6-4383-a6b7-ff388665d46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:51.669058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f787a1-73c6-4383-a6b7-ff388665d46a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:51.669336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f787a1-73c6-4383-a6b7-ff388665d46a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:51.669607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f787a1-73c6-4383-a6b7-ff388665d46a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:51.688637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f787a1-73c6-4383-a6b7-ff388665d46a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-37f787a1-73c6-4383-a6b7-ff388665d46a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:51.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:51.694398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:37:51.695361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 77/155] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:37:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:51.700470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f787a1-73c6-4383-a6b7-ff388665d46a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:51.732451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0bcf6-d350-467c-a9d6-42393bc26b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:51.733152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0bcf6-d350-467c-a9d6-42393bc26b62 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:51.733432 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0bcf6-d350-467c-a9d6-42393bc26b62 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:51.733705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0bcf6-d350-467c-a9d6-42393bc26b62 None admin] PATH_INFO: `/v3/users/a485edf4c68a4d0eaed5f79e91691cc5` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:51.734877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0bcf6-d350-467c-a9d6-42393bc26b62 None admin] RBAC: Authorizing `identity:get_user(user_id=a485edf4c68a4d0eaed5f79e91691cc5)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:51.736927 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0bcf6-d350-467c-a9d6-42393bc26b62 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:51.738515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 79/156] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:51 2022] GET /identity/v3/users/a485edf4c68a4d0eaed5f79e91691cc5 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:51.744562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0bcf6-d350-467c-a9d6-42393bc26b62 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:51.826092 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:51.847867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f787a1-73c6-4383-a6b7-ff388665d46a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:51.904408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa2656-99dd-4125-bdcc-937343220722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:51.905037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa2656-99dd-4125-bdcc-937343220722 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:51.905311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa2656-99dd-4125-bdcc-937343220722 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:51.905577 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa2656-99dd-4125-bdcc-937343220722 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:51.906254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa2656-99dd-4125-bdcc-937343220722 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:51.906554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa2656-99dd-4125-bdcc-937343220722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:51.908537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa2656-99dd-4125-bdcc-937343220722 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:51.917607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 78/157] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:51.928736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa2656-99dd-4125-bdcc-937343220722 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:51.962610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0bcf6-d350-467c-a9d6-42393bc26b62 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:52.020401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f8a41-84f8-4664-9b7c-e64e3e652aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:52.021029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f8a41-84f8-4664-9b7c-e64e3e652aa1 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:52.021305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f8a41-84f8-4664-9b7c-e64e3e652aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:52.021575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f8a41-84f8-4664-9b7c-e64e3e652aa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:52.036046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:52.104527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa2656-99dd-4125-bdcc-937343220722 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:52.110281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f8a41-84f8-4664-9b7c-e64e3e652aa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:52.111883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f8a41-84f8-4664-9b7c-e64e3e652aa1 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:52.176414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d281b1-45f6-4c68-b028-c4c4ec773d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:52.177073 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d281b1-45f6-4c68-b028-c4c4ec773d31 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:52.177352 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d281b1-45f6-4c68-b028-c4c4ec773d31 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:52.177623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d281b1-45f6-4c68-b028-c4c4ec773d31 None admin] PATH_INFO: `/v3/users/5064296cd28f403599a02675cb0c7103` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:52.178842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d281b1-45f6-4c68-b028-c4c4ec773d31 None admin] RBAC: Authorizing `identity:get_user(user_id=5064296cd28f403599a02675cb0c7103)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:52.180328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d281b1-45f6-4c68-b028-c4c4ec773d31 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:52.181942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 79/158] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:51 2022] GET /identity/v3/users/5064296cd28f403599a02675cb0c7103 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:52.187430 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d281b1-45f6-4c68-b028-c4c4ec773d31 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:52.215653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:52.254883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 80/159] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:37:52.260344 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f8a41-84f8-4664-9b7c-e64e3e652aa1 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:52.339846 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d281b1-45f6-4c68-b028-c4c4ec773d31 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:52.344315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:52.422855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a22edb-46c6-434a-8085-d7f58ad61db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:52.423504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a22edb-46c6-434a-8085-d7f58ad61db5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:52.423817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a22edb-46c6-434a-8085-d7f58ad61db5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:52.424086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a22edb-46c6-434a-8085-d7f58ad61db5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:52.471466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f8a41-84f8-4664-9b7c-e64e3e652aa1 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:52.519039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5ec3e-2744-416a-a573-18c9e8f3df60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:52.519697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5ec3e-2744-416a-a573-18c9e8f3df60 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:52.519970 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5ec3e-2744-416a-a573-18c9e8f3df60 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:52.520238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5ec3e-2744-416a-a573-18c9e8f3df60 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:52.521366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5ec3e-2744-416a-a573-18c9e8f3df60 None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:52.522802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5ec3e-2744-416a-a573-18c9e8f3df60 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:52.524635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 81/160] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:52 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:52.525627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104b669-bb4b-467a-bb9c-116b5791518d None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:52.525906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104b669-bb4b-467a-bb9c-116b5791518d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:52.526174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104b669-bb4b-467a-bb9c-116b5791518d None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:52.527601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 82/161] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:52.528638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfeecb-8222-4bd9-946f-3768d99b1033 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:52.528907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfeecb-8222-4bd9-946f-3768d99b1033 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:52.529171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfeecb-8222-4bd9-946f-3768d99b1033 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:52.530590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 83/162] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:52.531281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfeecb-8222-4bd9-946f-3768d99b1033 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:52.541226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a22edb-46c6-434a-8085-d7f58ad61db5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:52.542980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a22edb-46c6-434a-8085-d7f58ad61db5 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:52.572882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:52.638897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfeecb-8222-4bd9-946f-3768d99b1033 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:52.652326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:52.675142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b62760-a753-4fe2-89dc-d1420016722a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:52.675802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b62760-a753-4fe2-89dc-d1420016722a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:52.676077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b62760-a753-4fe2-89dc-d1420016722a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:52.676343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b62760-a753-4fe2-89dc-d1420016722a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:52.679706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b62760-a753-4fe2-89dc-d1420016722a None admin] RBAC: Adding query filter params (role.id=603573d2738b49c389ad913208d2ced1, scope.project.id=c37e716487e9421694a14be7fa5c2d0c, user.id=a485edf4c68a4d0eaed5f79e91691cc5) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:52.679706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b62760-a753-4fe2-89dc-d1420016722a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:52.679706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b62760-a753-4fe2-89dc-d1420016722a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:52.685700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 84/163] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:52 2022] GET /identity/v3/role_assignments?user.id=a485edf4c68a4d0eaed5f79e91691cc5&scope.project.id=c37e716487e9421694a14be7fa5c2d0c&role.id=603573d2738b49c389ad913208d2ced1 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:52.686949 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7a4a6-4bb2-471a-a819-07051698d37e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:52.687228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7a4a6-4bb2-471a-a819-07051698d37e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:52.687496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7a4a6-4bb2-471a-a819-07051698d37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:52.706135 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-7aa7a4a6-4bb2-471a-a819-07051698d37e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:52.707332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7aa7a4a6-4bb2-471a-a819-07051698d37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:52.743175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 80/164] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 14:37:52.745009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802600d-9817-4a84-a9de-0485fc3da7f6 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:52.745542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802600d-9817-4a84-a9de-0485fc3da7f6 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:52.746030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802600d-9817-4a84-a9de-0485fc3da7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:52.779020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 85/165] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:52.779924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7a4a6-4bb2-471a-a819-07051698d37e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:52.780640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a802600d-9817-4a84-a9de-0485fc3da7f6 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:52.782154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a802600d-9817-4a84-a9de-0485fc3da7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:52.885750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7a4a6-4bb2-471a-a819-07051698d37e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:52.896089 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:52.922350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93b7c9-6816-4bcc-8a52-85cf2e7a9565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:52.922988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93b7c9-6816-4bcc-8a52-85cf2e7a9565 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:52.923865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93b7c9-6816-4bcc-8a52-85cf2e7a9565 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:52.924136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93b7c9-6816-4bcc-8a52-85cf2e7a9565 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:52.925281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93b7c9-6816-4bcc-8a52-85cf2e7a9565 None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:52.926777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93b7c9-6816-4bcc-8a52-85cf2e7a9565 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:52.928813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 86/166] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:52 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:52.933472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 81/167] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:52.936791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93b7c9-6816-4bcc-8a52-85cf2e7a9565 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:53.059569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93b7c9-6816-4bcc-8a52-85cf2e7a9565 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:53.103910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a586696-2064-42c1-90e4-9a395bfac99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:53.104908 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a586696-2064-42c1-90e4-9a395bfac99f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:53.105403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a586696-2064-42c1-90e4-9a395bfac99f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:53.105866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a586696-2064-42c1-90e4-9a395bfac99f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:53.106837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a586696-2064-42c1-90e4-9a395bfac99f None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=c37e716487e9421694a14be7fa5c2d0c, user.id=5064296cd28f403599a02675cb0c7103) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:53.107331 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a586696-2064-42c1-90e4-9a395bfac99f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:53.109021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a586696-2064-42c1-90e4-9a395bfac99f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:53.116951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 87/168] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:52 2022] GET /identity/v3/role_assignments?user.id=5064296cd28f403599a02675cb0c7103&scope.project.id=c37e716487e9421694a14be7fa5c2d0c&role.id=9a439cf8eef34ada85428b378878e63e => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:53.293681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802600d-9817-4a84-a9de-0485fc3da7f6 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:53.337113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a586696-2064-42c1-90e4-9a395bfac99f None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:53.372229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:53.396163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:53.478898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802600d-9817-4a84-a9de-0485fc3da7f6 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:53.529205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a586696-2064-42c1-90e4-9a395bfac99f None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:53.555592 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e906f-0d9d-473c-8f8d-8521bead8847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:53.556598 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e906f-0d9d-473c-8f8d-8521bead8847 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:53.557048 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e906f-0d9d-473c-8f8d-8521bead8847 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:53.557486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e906f-0d9d-473c-8f8d-8521bead8847 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:53.577354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e906f-0d9d-473c-8f8d-8521bead8847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-7c5e906f-0d9d-473c-8f8d-8521bead8847 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:53.583263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:53.584343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:53.585105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:53.585105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:53.585105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:53.585105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:53.585105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:53.585105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:53.585105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:53.585105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:53.585105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:53.585105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:53.585105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:37:53.585105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:37:53.588433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 82/169] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:37:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:53.596920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e906f-0d9d-473c-8f8d-8521bead8847 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:53.602133 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c53da-b073-432a-b9fe-cc356f264981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:53.603917 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c53da-b073-432a-b9fe-cc356f264981 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:53.604337 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c53da-b073-432a-b9fe-cc356f264981 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:53.606795 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c53da-b073-432a-b9fe-cc356f264981 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:53.627150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c53da-b073-432a-b9fe-cc356f264981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-d07c53da-b073-432a-b9fe-cc356f264981 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:53.634198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:53.635278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:53.636267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:53.636267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:53.636267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:53.636267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:53.636267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:53.636267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:53.636267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:53.636267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:53.636267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:53.636267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:53.636267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:53.636267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:37:53.641007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 88/170] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:37:53.647538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c53da-b073-432a-b9fe-cc356f264981 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:53.765000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e906f-0d9d-473c-8f8d-8521bead8847 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:53.774002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:53.803381 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83ea30-634a-46e2-afc2-585e4dca6ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:53.804033 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83ea30-634a-46e2-afc2-585e4dca6ffe None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:53.804308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83ea30-634a-46e2-afc2-585e4dca6ffe None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:53.804923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83ea30-634a-46e2-afc2-585e4dca6ffe None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:53.805608 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83ea30-634a-46e2-afc2-585e4dca6ffe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:53.805913 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83ea30-634a-46e2-afc2-585e4dca6ffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:53.807423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83ea30-634a-46e2-afc2-585e4dca6ffe None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:53.816295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 83/171] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:37:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:53.823009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83ea30-634a-46e2-afc2-585e4dca6ffe None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:53.913593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:53.944119 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83ea30-634a-46e2-afc2-585e4dca6ffe None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:53.949664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c53da-b073-432a-b9fe-cc356f264981 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:53.981440 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efa53e-7008-4cb9-951d-8f7a06b99e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:53.982278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efa53e-7008-4cb9-951d-8f7a06b99e8f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:53.982654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efa53e-7008-4cb9-951d-8f7a06b99e8f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:53.983020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efa53e-7008-4cb9-951d-8f7a06b99e8f None admin] PATH_INFO: `/v3/users/a485edf4c68a4d0eaed5f79e91691cc5` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:53.984296 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efa53e-7008-4cb9-951d-8f7a06b99e8f None admin] RBAC: Authorizing `identity:get_user(user_id=a485edf4c68a4d0eaed5f79e91691cc5)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:53.985892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efa53e-7008-4cb9-951d-8f7a06b99e8f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:53.987888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 84/172] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:53 2022] GET /identity/v3/users/a485edf4c68a4d0eaed5f79e91691cc5 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:53.994899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efa53e-7008-4cb9-951d-8f7a06b99e8f None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:54.044408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826edd9a-2ae9-46e9-b262-4fa333da1732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:54.045002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:54.045115 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826edd9a-2ae9-46e9-b262-4fa333da1732 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:54.045309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826edd9a-2ae9-46e9-b262-4fa333da1732 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:54.045588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826edd9a-2ae9-46e9-b262-4fa333da1732 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:54.046275 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826edd9a-2ae9-46e9-b262-4fa333da1732 None admin] RBAC: Adding query filter params (name=member) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:54.046586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826edd9a-2ae9-46e9-b262-4fa333da1732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:54.048052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826edd9a-2ae9-46e9-b262-4fa333da1732 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:54.075319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 89/173] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:37:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:54.084708 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826edd9a-2ae9-46e9-b262-4fa333da1732 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:54.102357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efa53e-7008-4cb9-951d-8f7a06b99e8f None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:54.149809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe46a1b-cacd-4d37-848c-092bc91bff92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:54.150807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe46a1b-cacd-4d37-848c-092bc91bff92 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:54.151276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe46a1b-cacd-4d37-848c-092bc91bff92 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:54.156245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe46a1b-cacd-4d37-848c-092bc91bff92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:54.306130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826edd9a-2ae9-46e9-b262-4fa333da1732 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:54.311891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe46a1b-cacd-4d37-848c-092bc91bff92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:54.319706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe46a1b-cacd-4d37-848c-092bc91bff92 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:54.353516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1615d-d120-4077-8ed6-09c00b737162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:54.355838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1615d-d120-4077-8ed6-09c00b737162 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:54.356442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1615d-d120-4077-8ed6-09c00b737162 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:54.356910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1615d-d120-4077-8ed6-09c00b737162 None admin] PATH_INFO: `/v3/users/a485edf4c68a4d0eaed5f79e91691cc5` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:54.359261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1615d-d120-4077-8ed6-09c00b737162 None admin] RBAC: Authorizing `identity:get_user(user_id=a485edf4c68a4d0eaed5f79e91691cc5)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:54.361612 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1615d-d120-4077-8ed6-09c00b737162 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:54.363661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 90/174] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:54 2022] GET /identity/v3/users/a485edf4c68a4d0eaed5f79e91691cc5 => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:54.365906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:54.367006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650a1ec-0298-4013-b73b-0872d057b1e1 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:54.367504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650a1ec-0298-4013-b73b-0872d057b1e1 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:54.368004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650a1ec-0298-4013-b73b-0872d057b1e1 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:54.369636 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 91/175] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:54.371352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ade29-7d14-417a-936e-a22c6385b34c None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:54.371853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ade29-7d14-417a-936e-a22c6385b34c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:54.372500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ade29-7d14-417a-936e-a22c6385b34c None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:54.374653 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 92/176] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:54.375961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ade29-7d14-417a-936e-a22c6385b34c None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:54.420239 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:54.443934 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 85/177] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 14:37:54.445252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdaad4-f1f2-4ac6-b6a0-53924dcd0bdd None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:54.445536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdaad4-f1f2-4ac6-b6a0-53924dcd0bdd None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:54.445811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdaad4-f1f2-4ac6-b6a0-53924dcd0bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:54.464870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-efbdaad4-f1f2-4ac6-b6a0-53924dcd0bdd None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:54.466065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-efbdaad4-f1f2-4ac6-b6a0-53924dcd0bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:54.504610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ade29-7d14-417a-936e-a22c6385b34c None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:54.536482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 86/178] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:54.537766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e96972-8abe-4f53-bb14-a069f13b539e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:54.538049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e96972-8abe-4f53-bb14-a069f13b539e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:54.538319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e96972-8abe-4f53-bb14-a069f13b539e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:54.539645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e61cf-1d28-4d6f-957b-a50587b5b6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:54.540276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e61cf-1d28-4d6f-957b-a50587b5b6c9 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:54.540553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e61cf-1d28-4d6f-957b-a50587b5b6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:54.540823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e61cf-1d28-4d6f-957b-a50587b5b6c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:54.556291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-b3e96972-8abe-4f53-bb14-a069f13b539e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:54.557583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b3e96972-8abe-4f53-bb14-a069f13b539e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:54.621300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:54.630601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 87/179] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:54.631850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfa874-01b9-47be-abe2-5b0353ba36f5 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:54.632130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfa874-01b9-47be-abe2-5b0353ba36f5 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:54.632551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfa874-01b9-47be-abe2-5b0353ba36f5 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:54.634132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 88/180] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:54.636363 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e61cf-1d28-4d6f-957b-a50587b5b6c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:54.638147 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e61cf-1d28-4d6f-957b-a50587b5b6c9 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:54.638368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfa874-01b9-47be-abe2-5b0353ba36f5 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:54.669968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:54.761988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 93/181] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:37:54.762437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfa874-01b9-47be-abe2-5b0353ba36f5 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:54.764169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd53ac-9a56-4954-913c-acbbd5dfd457 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:54.764653 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd53ac-9a56-4954-913c-acbbd5dfd457 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:54.766020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd53ac-9a56-4954-913c-acbbd5dfd457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:54.787943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-87bd53ac-9a56-4954-913c-acbbd5dfd457 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:54.788965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-87bd53ac-9a56-4954-913c-acbbd5dfd457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:54.806669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13418a-88ae-4756-b0fd-3e42150a2abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:54.808114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13418a-88ae-4756-b0fd-3e42150a2abb None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:54.808797 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13418a-88ae-4756-b0fd-3e42150a2abb None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:54.809483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13418a-88ae-4756-b0fd-3e42150a2abb None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:54.811270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13418a-88ae-4756-b0fd-3e42150a2abb None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:54.813382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13418a-88ae-4756-b0fd-3e42150a2abb None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:54.815255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 89/182] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:54 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:54.816235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13418a-88ae-4756-b0fd-3e42150a2abb None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:54.895516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 94/183] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:54.895516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd53ac-9a56-4954-913c-acbbd5dfd457 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:54.902814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:54.959887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13418a-88ae-4756-b0fd-3e42150a2abb None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:54.971071 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:55.003831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0201f02-6736-4596-a981-e8e35dccbee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:55.004462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0201f02-6736-4596-a981-e8e35dccbee1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.004737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0201f02-6736-4596-a981-e8e35dccbee1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.005007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0201f02-6736-4596-a981-e8e35dccbee1 None admin] PATH_INFO: `/v3/projects/b247e15407e4415ab7ca3c594925d8c4` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.006213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0201f02-6736-4596-a981-e8e35dccbee1 None admin] RBAC: Authorizing `identity:get_project(project_id=b247e15407e4415ab7ca3c594925d8c4)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:55.007697 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0201f02-6736-4596-a981-e8e35dccbee1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:55.009481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 90/184] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:54 2022] GET /identity/v3/projects/b247e15407e4415ab7ca3c594925d8c4 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:55.010427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e50115-1a45-43ff-b52a-c8d96393fba2 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.010708 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e50115-1a45-43ff-b52a-c8d96393fba2 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.010984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e50115-1a45-43ff-b52a-c8d96393fba2 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.012373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 91/185] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:55.012871 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e50115-1a45-43ff-b52a-c8d96393fba2 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:55.026296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd53ac-9a56-4954-913c-acbbd5dfd457 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:55.064230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d6ff5-ab80-4a11-ac93-89f257f3413c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:55.064932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d6ff5-ab80-4a11-ac93-89f257f3413c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.065105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d6ff5-ab80-4a11-ac93-89f257f3413c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.065366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d6ff5-ab80-4a11-ac93-89f257f3413c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.066083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d6ff5-ab80-4a11-ac93-89f257f3413c None admin] RBAC: Adding query filter params (role.id=9df5a7d7691c49b9a78f0db0a050b7fe, scope.project.id=c37e716487e9421694a14be7fa5c2d0c, user.id=a485edf4c68a4d0eaed5f79e91691cc5) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:55.066384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d6ff5-ab80-4a11-ac93-89f257f3413c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:55.067832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d6ff5-ab80-4a11-ac93-89f257f3413c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:55.073700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 95/186] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:54 2022] GET /identity/v3/role_assignments?user.id=a485edf4c68a4d0eaed5f79e91691cc5&scope.project.id=c37e716487e9421694a14be7fa5c2d0c&role.id=9df5a7d7691c49b9a78f0db0a050b7fe => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:55.074431 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d6ff5-ab80-4a11-ac93-89f257f3413c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:55.141728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e50115-1a45-43ff-b52a-c8d96393fba2 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:55.175940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:55.183835 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d61c9-f499-4823-9834-3a39fbc227e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:55.184522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d61c9-f499-4823-9834-3a39fbc227e5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.184801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d61c9-f499-4823-9834-3a39fbc227e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.185072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d61c9-f499-4823-9834-3a39fbc227e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.196998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d61c9-f499-4823-9834-3a39fbc227e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-9b8d61c9-f499-4823-9834-3a39fbc227e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:55.201808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:55.202719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:37:55.203551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 92/187] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:37:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:55.204310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688371ce-bcbe-40c8-bf06-b95b47bb21c9 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.205045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688371ce-bcbe-40c8-bf06-b95b47bb21c9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.205045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688371ce-bcbe-40c8-bf06-b95b47bb21c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.214946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d6ff5-ab80-4a11-ac93-89f257f3413c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:55.226517 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-688371ce-bcbe-40c8-bf06-b95b47bb21c9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:55.228028 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-688371ce-bcbe-40c8-bf06-b95b47bb21c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:55.232499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:55.252091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba35cf-e2d0-4384-9fbc-7a02eefff67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:55.252727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba35cf-e2d0-4384-9fbc-7a02eefff67c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.253007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba35cf-e2d0-4384-9fbc-7a02eefff67c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.253281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba35cf-e2d0-4384-9fbc-7a02eefff67c None admin] PATH_INFO: `/v3/domains/default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.254392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba35cf-e2d0-4384-9fbc-7a02eefff67c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:55.255899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba35cf-e2d0-4384-9fbc-7a02eefff67c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:55.257492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 96/188] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:55.262821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba35cf-e2d0-4384-9fbc-7a02eefff67c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:55.307191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 93/189] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:55.308125 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688371ce-bcbe-40c8-bf06-b95b47bb21c9 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:55.365859 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba35cf-e2d0-4384-9fbc-7a02eefff67c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:55.398761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fbca4-dc35-4588-aa20-9814f3151b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:55.399358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fbca4-dc35-4588-aa20-9814f3151b7a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.399634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fbca4-dc35-4588-aa20-9814f3151b7a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.399973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fbca4-dc35-4588-aa20-9814f3151b7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.400697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5fbca4-dc35-4588-aa20-9814f3151b7a None admin] RBAC: Adding query filter params (role.id=603573d2738b49c389ad913208d2ced1, scope.project.id=b247e15407e4415ab7ca3c594925d8c4, user.id=a485edf4c68a4d0eaed5f79e91691cc5) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:55.401000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5fbca4-dc35-4588-aa20-9814f3151b7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:55.402427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5fbca4-dc35-4588-aa20-9814f3151b7a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:55.408684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 97/190] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:55 2022] GET /identity/v3/role_assignments?user.id=a485edf4c68a4d0eaed5f79e91691cc5&scope.project.id=b247e15407e4415ab7ca3c594925d8c4&role.id=603573d2738b49c389ad913208d2ced1 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:55.414970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688371ce-bcbe-40c8-bf06-b95b47bb21c9 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:55.416675 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fbca4-dc35-4588-aa20-9814f3151b7a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:55.448931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:55.463654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de12492-94c4-4306-bba7-20005a0c3566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:55.464438 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de12492-94c4-4306-bba7-20005a0c3566 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.464799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de12492-94c4-4306-bba7-20005a0c3566 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.465128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de12492-94c4-4306-bba7-20005a0c3566 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.478137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de12492-94c4-4306-bba7-20005a0c3566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:55.482106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-1de12492-94c4-4306-bba7-20005a0c3566 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:55.482988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:55.484460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:37:55.485337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 94/191] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:37:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:55.489946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de12492-94c4-4306-bba7-20005a0c3566 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:55.545923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fbca4-dc35-4588-aa20-9814f3151b7a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:55.584599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850d817-47e5-47a8-b78c-0f1031f7b568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:55.585222 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850d817-47e5-47a8-b78c-0f1031f7b568 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.585483 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850d817-47e5-47a8-b78c-0f1031f7b568 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.585752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850d817-47e5-47a8-b78c-0f1031f7b568 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.586419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850d817-47e5-47a8-b78c-0f1031f7b568 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:55.586720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850d817-47e5-47a8-b78c-0f1031f7b568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:55.588142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850d817-47e5-47a8-b78c-0f1031f7b568 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:55.597557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 98/192] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:55.598236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850d817-47e5-47a8-b78c-0f1031f7b568 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:55.614264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de12492-94c4-4306-bba7-20005a0c3566 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:55.657008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7c410-8a52-4f08-a4cc-4a22a6a4912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:55.658148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7c410-8a52-4f08-a4cc-4a22a6a4912c None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.658422 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7c410-8a52-4f08-a4cc-4a22a6a4912c None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.658696 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7c410-8a52-4f08-a4cc-4a22a6a4912c None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.659334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7c410-8a52-4f08-a4cc-4a22a6a4912c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:55.660834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7c410-8a52-4f08-a4cc-4a22a6a4912c None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:55.682123 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 95/193] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:37:55 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:37:55.691903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7c410-8a52-4f08-a4cc-4a22a6a4912c None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:55.711502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850d817-47e5-47a8-b78c-0f1031f7b568 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:55.726939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:55.730488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:55.754355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de775e97-d73b-49c1-ac2d-09a56882199b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:55.754943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de775e97-d73b-49c1-ac2d-09a56882199b None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.755217 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de775e97-d73b-49c1-ac2d-09a56882199b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.755497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de775e97-d73b-49c1-ac2d-09a56882199b None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.756197 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de775e97-d73b-49c1-ac2d-09a56882199b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:55.756499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de775e97-d73b-49c1-ac2d-09a56882199b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:55.757915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de775e97-d73b-49c1-ac2d-09a56882199b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:55.766891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 99/194] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:37:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:55.771750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de775e97-d73b-49c1-ac2d-09a56882199b None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:55.800390 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7c410-8a52-4f08-a4cc-4a22a6a4912c None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:55.838283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb245c-e3ed-43ff-91f4-f45e188864b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:55.838931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb245c-e3ed-43ff-91f4-f45e188864b4 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.839211 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb245c-e3ed-43ff-91f4-f45e188864b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.839489 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb245c-e3ed-43ff-91f4-f45e188864b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.853016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb245c-e3ed-43ff-91f4-f45e188864b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-73bb245c-e3ed-43ff-91f4-f45e188864b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:55.858337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:55.859221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:55.860113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:55.860113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:55.860113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:55.860113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:55.860113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:55.860113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:55.860113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:55.860113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:55.860113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:55.860113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:55.860113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:37:55.860113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:37:55.863813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 96/195] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:55.865296 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb245c-e3ed-43ff-91f4-f45e188864b4 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:55.886776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de775e97-d73b-49c1-ac2d-09a56882199b None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:55.938692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48fbd7-6184-4e8e-81ad-f3dbb347aa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:55.939338 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48fbd7-6184-4e8e-81ad-f3dbb347aa04 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:55.939615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48fbd7-6184-4e8e-81ad-f3dbb347aa04 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:55.939950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48fbd7-6184-4e8e-81ad-f3dbb347aa04 None admin] PATH_INFO: `/v3/users/10b61d42978349cfba564d042f7dcdb9` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:55.961509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48fbd7-6184-4e8e-81ad-f3dbb347aa04 None admin] RBAC: Authorizing `identity:get_user(user_id=10b61d42978349cfba564d042f7dcdb9)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:55.963285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48fbd7-6184-4e8e-81ad-f3dbb347aa04 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:55.965826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 100/196] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:55 2022] GET /identity/v3/users/10b61d42978349cfba564d042f7dcdb9 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:55.966698 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48fbd7-6184-4e8e-81ad-f3dbb347aa04 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:55.978617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:55.990502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb245c-e3ed-43ff-91f4-f45e188864b4 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:56.005557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:56.037087 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1fce0-fe8b-401f-a10b-369964ac27e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:56.038090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1fce0-fe8b-401f-a10b-369964ac27e0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:56.038543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1fce0-fe8b-401f-a10b-369964ac27e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:56.039003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1fce0-fe8b-401f-a10b-369964ac27e0 None admin] PATH_INFO: `/v3/users/10b61d42978349cfba564d042f7dcdb9` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:56.040604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1fce0-fe8b-401f-a10b-369964ac27e0 None admin] RBAC: Authorizing `identity:get_user(user_id=10b61d42978349cfba564d042f7dcdb9)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:56.042354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1fce0-fe8b-401f-a10b-369964ac27e0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:56.044763 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 97/197] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:55 2022] GET /identity/v3/users/10b61d42978349cfba564d042f7dcdb9 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:56.045902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1fce0-fe8b-401f-a10b-369964ac27e0 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:56.090920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48fbd7-6184-4e8e-81ad-f3dbb347aa04 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:56.128915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c5e91-0bff-4ca9-882c-998e7caed9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:56.129816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c5e91-0bff-4ca9-882c-998e7caed9e9 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:56.130265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c5e91-0bff-4ca9-882c-998e7caed9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:56.130707 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c5e91-0bff-4ca9-882c-998e7caed9e9 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:56.131566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c5e91-0bff-4ca9-882c-998e7caed9e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:56.132105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c5e91-0bff-4ca9-882c-998e7caed9e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:56.133653 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c5e91-0bff-4ca9-882c-998e7caed9e9 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:56.143657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 101/198] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:37:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:56.144740 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c5e91-0bff-4ca9-882c-998e7caed9e9 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:56.153054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1fce0-fe8b-401f-a10b-369964ac27e0 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:56.189131 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6ea8b-50a0-4245-92b4-edd255910b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:56.189721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6ea8b-50a0-4245-92b4-edd255910b24 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:56.190067 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6ea8b-50a0-4245-92b4-edd255910b24 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:56.190405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6ea8b-50a0-4245-92b4-edd255910b24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:56.225725 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:56.246478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:56.255698 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c5e91-0bff-4ca9-882c-998e7caed9e9 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:56.264665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6ea8b-50a0-4245-92b4-edd255910b24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:56.266539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6ea8b-50a0-4245-92b4-edd255910b24 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:56.293725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825da272-294d-4684-bf6d-006fd0154bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:56.294654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825da272-294d-4684-bf6d-006fd0154bd8 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:56.295105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825da272-294d-4684-bf6d-006fd0154bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:56.295544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825da272-294d-4684-bf6d-006fd0154bd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:56.386377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825da272-294d-4684-bf6d-006fd0154bd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:56.386377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825da272-294d-4684-bf6d-006fd0154bd8 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:56.387169 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 98/199] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:37:56.389193 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6ea8b-50a0-4245-92b4-edd255910b24 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:56.475080 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:56.502973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:56.516231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 102/200] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:37:56.517768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825da272-294d-4684-bf6d-006fd0154bd8 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:56.525207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6ea8b-50a0-4245-92b4-edd255910b24 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:56.572110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757775fe-774e-4c40-b597-65ca8e79c898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:56.572838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757775fe-774e-4c40-b597-65ca8e79c898 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:56.573180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757775fe-774e-4c40-b597-65ca8e79c898 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:56.573675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757775fe-774e-4c40-b597-65ca8e79c898 None admin] PATH_INFO: `/v3/users/c9c4f75f67d84d6d866bed009c7bbc03` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:56.575262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757775fe-774e-4c40-b597-65ca8e79c898 None admin] RBAC: Authorizing `identity:get_user(user_id=c9c4f75f67d84d6d866bed009c7bbc03)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:56.577508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757775fe-774e-4c40-b597-65ca8e79c898 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:56.580272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 99/201] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:56 2022] GET /identity/v3/users/c9c4f75f67d84d6d866bed009c7bbc03 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:56.581843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757775fe-774e-4c40-b597-65ca8e79c898 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:56.640953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825da272-294d-4684-bf6d-006fd0154bd8 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:56.686908 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757775fe-774e-4c40-b597-65ca8e79c898 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:56.689856 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff281d9-e8f5-4fb8-bf8e-d8b92e786285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:56.691449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff281d9-e8f5-4fb8-bf8e-d8b92e786285 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:56.691855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff281d9-e8f5-4fb8-bf8e-d8b92e786285 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:56.692223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff281d9-e8f5-4fb8-bf8e-d8b92e786285 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:56.693452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff281d9-e8f5-4fb8-bf8e-d8b92e786285 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:56.694996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff281d9-e8f5-4fb8-bf8e-d8b92e786285 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:56.702226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 103/202] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:56 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:56.702226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff281d9-e8f5-4fb8-bf8e-d8b92e786285 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:56.729539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0f0e8-a770-4fb1-bdca-78e49bb595be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:56.730614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0f0e8-a770-4fb1-bdca-78e49bb595be None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:56.731064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0f0e8-a770-4fb1-bdca-78e49bb595be None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:56.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0f0e8-a770-4fb1-bdca-78e49bb595be None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:56.732850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0f0e8-a770-4fb1-bdca-78e49bb595be None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:56.734468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0f0e8-a770-4fb1-bdca-78e49bb595be None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:56.736827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 100/203] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:56 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:56.737836 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0f0e8-a770-4fb1-bdca-78e49bb595be None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:56.740679 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:56.789962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:56.836975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff281d9-e8f5-4fb8-bf8e-d8b92e786285 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:56.845000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0f0e8-a770-4fb1-bdca-78e49bb595be None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:56.878245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758db3b-83fa-4245-becc-b3a32d9afc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:56.878870 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758db3b-83fa-4245-becc-b3a32d9afc29 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:56.879166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758db3b-83fa-4245-becc-b3a32d9afc29 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:56.879436 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758db3b-83fa-4245-becc-b3a32d9afc29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:56.902924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972c977-157d-4970-9c0a-f62ecb2e6a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:56.903660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972c977-157d-4970-9c0a-f62ecb2e6a99 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:56.904137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972c977-157d-4970-9c0a-f62ecb2e6a99 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:56.904505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972c977-157d-4970-9c0a-f62ecb2e6a99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:56.905400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972c977-157d-4970-9c0a-f62ecb2e6a99 None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=c9ba9949b27349ab83f686eee37fe6f7, user.id=10b61d42978349cfba564d042f7dcdb9) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:56.905847 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972c977-157d-4970-9c0a-f62ecb2e6a99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:56.907503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972c977-157d-4970-9c0a-f62ecb2e6a99 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:56.913633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 101/204] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:56 2022] GET /identity/v3/role_assignments?user.id=10b61d42978349cfba564d042f7dcdb9&scope.project.id=c9ba9949b27349ab83f686eee37fe6f7&role.id=9a439cf8eef34ada85428b378878e63e => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:56.914594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972c977-157d-4970-9c0a-f62ecb2e6a99 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:57.009412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758db3b-83fa-4245-becc-b3a32d9afc29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:57.011118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758db3b-83fa-4245-becc-b3a32d9afc29 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:57.040071 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:57.077212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:57.109795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972c977-157d-4970-9c0a-f62ecb2e6a99 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:57.120454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 104/205] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:37:57.128802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758db3b-83fa-4245-becc-b3a32d9afc29 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:57.167445 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69b7d1-aeae-4378-9756-a5a09ad97289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:57.168032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69b7d1-aeae-4378-9756-a5a09ad97289 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:57.168306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69b7d1-aeae-4378-9756-a5a09ad97289 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:57.168574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69b7d1-aeae-4378-9756-a5a09ad97289 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:57.169303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69b7d1-aeae-4378-9756-a5a09ad97289 None admin] RBAC: Adding query filter params (role.id=9df5a7d7691c49b9a78f0db0a050b7fe, scope.project.id=c9ba9949b27349ab83f686eee37fe6f7, user.id=10b61d42978349cfba564d042f7dcdb9) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:57.169607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69b7d1-aeae-4378-9756-a5a09ad97289 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:57.171182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69b7d1-aeae-4378-9756-a5a09ad97289 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:57.178145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 102/206] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:56 2022] GET /identity/v3/role_assignments?user.id=10b61d42978349cfba564d042f7dcdb9&scope.project.id=c9ba9949b27349ab83f686eee37fe6f7&role.id=9df5a7d7691c49b9a78f0db0a050b7fe => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:57.244666 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758db3b-83fa-4245-becc-b3a32d9afc29 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:57.294655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e13fa1-29ee-4474-acb3-8b0bf1e36e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:57.296156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e13fa1-29ee-4474-acb3-8b0bf1e36e60 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:57.296610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e13fa1-29ee-4474-acb3-8b0bf1e36e60 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:57.297049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e13fa1-29ee-4474-acb3-8b0bf1e36e60 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:57.300424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e13fa1-29ee-4474-acb3-8b0bf1e36e60 None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:57.302225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e13fa1-29ee-4474-acb3-8b0bf1e36e60 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:57.304508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 105/207] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:57 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:57.310736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69b7d1-aeae-4378-9756-a5a09ad97289 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:57.421257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69b7d1-aeae-4378-9756-a5a09ad97289 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:57.450322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:57.457969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632df6b-a973-4407-b9ca-d0415c3ea54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:57.459010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632df6b-a973-4407-b9ca-d0415c3ea54a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:57.459348 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632df6b-a973-4407-b9ca-d0415c3ea54a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:57.459623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632df6b-a973-4407-b9ca-d0415c3ea54a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:57.460380 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632df6b-a973-4407-b9ca-d0415c3ea54a None admin] RBAC: Adding query filter params (role.id=a853ef2aaba2459f86b50cae95d23577, scope.project.id=c37e716487e9421694a14be7fa5c2d0c, user.id=c9c4f75f67d84d6d866bed009c7bbc03) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:57.460687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632df6b-a973-4407-b9ca-d0415c3ea54a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:57.462290 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632df6b-a973-4407-b9ca-d0415c3ea54a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:57.468350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 103/208] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:57 2022] GET /identity/v3/role_assignments?user.id=c9c4f75f67d84d6d866bed009c7bbc03&scope.project.id=c37e716487e9421694a14be7fa5c2d0c&role.id=a853ef2aaba2459f86b50cae95d23577 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:57.827189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e0b1-a364-40fa-a40a-2efbb9e24aa4 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:57.827507 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e0b1-a364-40fa-a40a-2efbb9e24aa4 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:57.827797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e0b1-a364-40fa-a40a-2efbb9e24aa4 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:57.829304 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 106/209] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:57.834682 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238db69-dac4-4f34-b604-8c7924922381 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:57.835112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238db69-dac4-4f34-b604-8c7924922381 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:57.835293 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238db69-dac4-4f34-b604-8c7924922381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:57.854218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-b238db69-dac4-4f34-b604-8c7924922381 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:57.855477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b238db69-dac4-4f34-b604-8c7924922381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:57.934151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 104/210] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:58.068517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c37388-663d-4cec-976a-23591b6a247c None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:58.068808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c37388-663d-4cec-976a-23591b6a247c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:58.069057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c37388-663d-4cec-976a-23591b6a247c None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:58.070671 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 107/211] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:58.076011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3864b0a-0f31-456a-954d-a5d6e4777ef0 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:58.076292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3864b0a-0f31-456a-954d-a5d6e4777ef0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:58.076563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3864b0a-0f31-456a-954d-a5d6e4777ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:58.095391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a3864b0a-0f31-456a-954d-a5d6e4777ef0 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:58.096526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a3864b0a-0f31-456a-954d-a5d6e4777ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:58.164613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 105/212] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:58.222751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c37388-663d-4cec-976a-23591b6a247c None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:58.254768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:58.339708 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c37388-663d-4cec-976a-23591b6a247c None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:58.383463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa52aa-af12-4eaa-9fbd-ab4cc4ac2e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:58.384567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa52aa-af12-4eaa-9fbd-ab4cc4ac2e38 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:58.385094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa52aa-af12-4eaa-9fbd-ab4cc4ac2e38 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:58.385575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa52aa-af12-4eaa-9fbd-ab4cc4ac2e38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:58.401474 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa52aa-af12-4eaa-9fbd-ab4cc4ac2e38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-14fa52aa-af12-4eaa-9fbd-ab4cc4ac2e38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:58.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:58.408589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:58.410155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:58.410155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:58.410155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:58.410155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:58.410155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:58.410155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:58.410155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:58.410155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:58.410155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:58.410155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:58.410155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:37:58.410155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:37:58.412184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 108/213] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:58.414627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3864b0a-0f31-456a-954d-a5d6e4777ef0 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:58.468623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa52aa-af12-4eaa-9fbd-ab4cc4ac2e38 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:58.508146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:58.560971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3864b0a-0f31-456a-954d-a5d6e4777ef0 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:58.604341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b85def-2420-4828-b1eb-8150943e2846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:58.604960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b85def-2420-4828-b1eb-8150943e2846 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:58.605374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b85def-2420-4828-b1eb-8150943e2846 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:58.605840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b85def-2420-4828-b1eb-8150943e2846 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:58.606517 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b85def-2420-4828-b1eb-8150943e2846 None admin] RBAC: Adding query filter params (name=member) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:58.606821 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b85def-2420-4828-b1eb-8150943e2846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:58.607006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:58.608373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b85def-2420-4828-b1eb-8150943e2846 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:58.620046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa52aa-af12-4eaa-9fbd-ab4cc4ac2e38 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:58.621744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 106/214] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:37:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:58.628353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b85def-2420-4828-b1eb-8150943e2846 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:58.661417 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14260e1-f87a-4b6a-ac63-2fcdc8ef8bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:58.662065 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14260e1-f87a-4b6a-ac63-2fcdc8ef8bd4 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:58.662412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14260e1-f87a-4b6a-ac63-2fcdc8ef8bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:58.662750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14260e1-f87a-4b6a-ac63-2fcdc8ef8bd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:58.663960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14260e1-f87a-4b6a-ac63-2fcdc8ef8bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:58.665499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14260e1-f87a-4b6a-ac63-2fcdc8ef8bd4 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:58.667379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 109/215] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:37:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:58.672633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14260e1-f87a-4b6a-ac63-2fcdc8ef8bd4 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:58.738151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b85def-2420-4828-b1eb-8150943e2846 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:58.756373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:58.775599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14260e1-f87a-4b6a-ac63-2fcdc8ef8bd4 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:58.778150 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024955f-d722-437f-939a-5887c5f7abe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:58.778793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024955f-d722-437f-939a-5887c5f7abe7 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:58.779070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024955f-d722-437f-939a-5887c5f7abe7 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:58.779344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024955f-d722-437f-939a-5887c5f7abe7 None admin] PATH_INFO: `/v3/users/ea39ac4cd0c644e79a6328aa5ae3e4d4` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:58.794048 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024955f-d722-437f-939a-5887c5f7abe7 None admin] RBAC: Authorizing `identity:get_user(user_id=ea39ac4cd0c644e79a6328aa5ae3e4d4)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:58.795527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024955f-d722-437f-939a-5887c5f7abe7 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:58.797630 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 107/216] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:37:58 2022] GET /identity/v3/users/ea39ac4cd0c644e79a6328aa5ae3e4d4 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:37:58.803021 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024955f-d722-437f-939a-5887c5f7abe7 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:58.814385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1c7d3-0d16-473c-8d4e-a1ea6ef08765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:58.815110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1c7d3-0d16-473c-8d4e-a1ea6ef08765 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:58.815501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1c7d3-0d16-473c-8d4e-a1ea6ef08765 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:58.815802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1c7d3-0d16-473c-8d4e-a1ea6ef08765 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:58.816509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1c7d3-0d16-473c-8d4e-a1ea6ef08765 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:58.818123 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1c7d3-0d16-473c-8d4e-a1ea6ef08765 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:58.844044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 110/217] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:37:58 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:37:58.921216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024955f-d722-437f-939a-5887c5f7abe7 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:58.958465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215d4ee-0b85-4e4f-8d0e-15cd11d3a21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:58.959088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215d4ee-0b85-4e4f-8d0e-15cd11d3a21a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:58.959362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215d4ee-0b85-4e4f-8d0e-15cd11d3a21a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:58.959631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215d4ee-0b85-4e4f-8d0e-15cd11d3a21a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:59.007445 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:59.034260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215d4ee-0b85-4e4f-8d0e-15cd11d3a21a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:59.035750 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215d4ee-0b85-4e4f-8d0e-15cd11d3a21a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:59.141556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 108/218] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:37:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 14:37:59.147536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1c7d3-0d16-473c-8d4e-a1ea6ef08765 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:59.165458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:59.259532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1c7d3-0d16-473c-8d4e-a1ea6ef08765 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:59.284503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad9710-a060-41b0-aed6-e66925de76c8 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:59.285465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad9710-a060-41b0-aed6-e66925de76c8 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:59.286209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad9710-a060-41b0-aed6-e66925de76c8 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:59.288819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 109/219] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:59.294028 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19477b90-b787-4723-8f1b-47a8adcd8215 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:59.294662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19477b90-b787-4723-8f1b-47a8adcd8215 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:59.295124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19477b90-b787-4723-8f1b-47a8adcd8215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:59.296701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f383882-7897-4d8f-9092-28869d837250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:59.297335 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f383882-7897-4d8f-9092-28869d837250 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:59.299270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f383882-7897-4d8f-9092-28869d837250 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:59.299270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f383882-7897-4d8f-9092-28869d837250 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:59.299270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f383882-7897-4d8f-9092-28869d837250 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:59.300489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f383882-7897-4d8f-9092-28869d837250 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:59.302114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 111/220] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:37:59 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:59.303348 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78b7b1-0eed-4fee-ae05-0559ff07ad86 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:59.303629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78b7b1-0eed-4fee-ae05-0559ff07ad86 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:59.303922 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78b7b1-0eed-4fee-ae05-0559ff07ad86 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:59.305229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 112/221] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:37:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:37:59.308559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78b7b1-0eed-4fee-ae05-0559ff07ad86 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:59.320795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-19477b90-b787-4723-8f1b-47a8adcd8215 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:59.322386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-19477b90-b787-4723-8f1b-47a8adcd8215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:59.408905 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:59.420582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 110/222] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:59.427729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822c79f-e54d-4de3-bf10-42af2a63aa81 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:59.427729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822c79f-e54d-4de3-bf10-42af2a63aa81 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:59.427729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822c79f-e54d-4de3-bf10-42af2a63aa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:59.441087 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:37:59.441913 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78b7b1-0eed-4fee-ae05-0559ff07ad86 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:59.463789 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a822c79f-e54d-4de3-bf10-42af2a63aa81 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:37:59.465056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a822c79f-e54d-4de3-bf10-42af2a63aa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:37:59.494677 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f030bb-b57f-4ba6-9cdd-98a3c20eb1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:59.495595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f030bb-b57f-4ba6-9cdd-98a3c20eb1fa None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:59.496042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f030bb-b57f-4ba6-9cdd-98a3c20eb1fa None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:59.496460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f030bb-b57f-4ba6-9cdd-98a3c20eb1fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:59.497337 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f030bb-b57f-4ba6-9cdd-98a3c20eb1fa None admin] RBAC: Adding query filter params (role.id=603573d2738b49c389ad913208d2ced1, scope.project.id=c9ba9949b27349ab83f686eee37fe6f7, user.id=ea39ac4cd0c644e79a6328aa5ae3e4d4) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:37:59.498109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f030bb-b57f-4ba6-9cdd-98a3c20eb1fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:37:59.500847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f030bb-b57f-4ba6-9cdd-98a3c20eb1fa None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:37:59.512877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 113/223] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:37:59 2022] GET /identity/v3/role_assignments?user.id=ea39ac4cd0c644e79a6328aa5ae3e4d4&scope.project.id=c9ba9949b27349ab83f686eee37fe6f7&role.id=603573d2738b49c389ad913208d2ced1 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:37:59.559561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 111/224] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:37:59.774857 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f030bb-b57f-4ba6-9cdd-98a3c20eb1fa None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:59.899713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f030bb-b57f-4ba6-9cdd-98a3c20eb1fa None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:37:59.945281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd0262-f0c0-49b3-a596-3a11adee71be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:37:59.945907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd0262-f0c0-49b3-a596-3a11adee71be None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:37:59.946181 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd0262-f0c0-49b3-a596-3a11adee71be None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:37:59.946447 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd0262-f0c0-49b3-a596-3a11adee71be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:37:59.963248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd0262-f0c0-49b3-a596-3a11adee71be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-b9dd0262-f0c0-49b3-a596-3a11adee71be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:37:59.968403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:37:59.969280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:37:59.970106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:37:59.970106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:37:59.970106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:37:59.970106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:37:59.970106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:37:59.970106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:37:59.970106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:37:59.970106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:37:59.970106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:37:59.970106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:37:59.970106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:37:59.970106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:37:59.970711 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 114/225] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:37:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:37:59.973493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822c79f-e54d-4de3-bf10-42af2a63aa81 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:37:59.984992 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd0262-f0c0-49b3-a596-3a11adee71be None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:00.004552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:00.089585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822c79f-e54d-4de3-bf10-42af2a63aa81 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:00.127139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc91ba5-b6c7-44ff-9ad4-87ecb4be890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:00.128081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc91ba5-b6c7-44ff-9ad4-87ecb4be890a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:00.128513 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc91ba5-b6c7-44ff-9ad4-87ecb4be890a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:00.128921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc91ba5-b6c7-44ff-9ad4-87ecb4be890a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:00.130918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:00.148591 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc91ba5-b6c7-44ff-9ad4-87ecb4be890a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:00.151110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd0262-f0c0-49b3-a596-3a11adee71be None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-afc91ba5-b6c7-44ff-9ad4-87ecb4be890a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:00.158369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:00.159467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:00.160396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:00.160396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:00.160396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:00.160396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:00.160396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:00.160396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:00.160396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:00.160396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:00.160396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:00.160396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:00.160396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:00.160396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:00.164043 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 112/226] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:37:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:00.165581 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc91ba5-b6c7-44ff-9ad4-87ecb4be890a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:00.191980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0dbc6-8585-48f7-b37e-99a8340eccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:00.192586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0dbc6-8585-48f7-b37e-99a8340eccd6 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:00.193025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0dbc6-8585-48f7-b37e-99a8340eccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:00.193441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0dbc6-8585-48f7-b37e-99a8340eccd6 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:00.194335 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0dbc6-8585-48f7-b37e-99a8340eccd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:00.194866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0dbc6-8585-48f7-b37e-99a8340eccd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:00.196445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0dbc6-8585-48f7-b37e-99a8340eccd6 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:00.207165 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 115/227] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:37:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:00.212958 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0dbc6-8585-48f7-b37e-99a8340eccd6 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:00.268609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc91ba5-b6c7-44ff-9ad4-87ecb4be890a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:00.303828 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8da368-61b6-4655-8e10-47a8c56c26f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:00.304435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da368-61b6-4655-8e10-47a8c56c26f9 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:00.304709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da368-61b6-4655-8e10-47a8c56c26f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:00.304976 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da368-61b6-4655-8e10-47a8c56c26f9 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:00.305637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8da368-61b6-4655-8e10-47a8c56c26f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:00.305935 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8da368-61b6-4655-8e10-47a8c56c26f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:00.311093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8da368-61b6-4655-8e10-47a8c56c26f9 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:00.319796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 113/228] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:00.321475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8da368-61b6-4655-8e10-47a8c56c26f9 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:00.349221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:00.396183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:00.429714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8da368-61b6-4655-8e10-47a8c56c26f9 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:00.448840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0dbc6-8585-48f7-b37e-99a8340eccd6 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:00.471129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b0c7-fe18-4504-8034-ed809a8b1fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:00.471793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b0c7-fe18-4504-8034-ed809a8b1fe5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:00.472069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b0c7-fe18-4504-8034-ed809a8b1fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:00.472337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b0c7-fe18-4504-8034-ed809a8b1fe5 None admin] PATH_INFO: `/v3/users/10b61d42978349cfba564d042f7dcdb9` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:00.473623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819b0c7-fe18-4504-8034-ed809a8b1fe5 None admin] RBAC: Authorizing `identity:get_user(user_id=10b61d42978349cfba564d042f7dcdb9)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:00.475059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819b0c7-fe18-4504-8034-ed809a8b1fe5 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:00.476661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 114/229] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:00 2022] GET /identity/v3/users/10b61d42978349cfba564d042f7dcdb9 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:00.484119 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b0c7-fe18-4504-8034-ed809a8b1fe5 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:00.527340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66426048-9837-498b-9d4a-c466825275ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:00.527994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66426048-9837-498b-9d4a-c466825275ff None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:00.528264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66426048-9837-498b-9d4a-c466825275ff None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:00.528529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66426048-9837-498b-9d4a-c466825275ff None admin] PATH_INFO: `/v3/users/10b61d42978349cfba564d042f7dcdb9` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:00.529646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66426048-9837-498b-9d4a-c466825275ff None admin] RBAC: Authorizing `identity:get_user(user_id=10b61d42978349cfba564d042f7dcdb9)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:00.531260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66426048-9837-498b-9d4a-c466825275ff None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:00.534093 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 116/230] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:00 2022] GET /identity/v3/users/10b61d42978349cfba564d042f7dcdb9 => generated 283 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:00.538999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66426048-9837-498b-9d4a-c466825275ff None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:00.668017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b0c7-fe18-4504-8034-ed809a8b1fe5 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:00.716043 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:00.730485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66426048-9837-498b-9d4a-c466825275ff None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:00.733627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937baefb-3766-4262-81d6-0a414217de7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:00.734266 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937baefb-3766-4262-81d6-0a414217de7b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:00.734541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937baefb-3766-4262-81d6-0a414217de7b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:00.734811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937baefb-3766-4262-81d6-0a414217de7b None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:00.737002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937baefb-3766-4262-81d6-0a414217de7b None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:00.738466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937baefb-3766-4262-81d6-0a414217de7b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:00.749905 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 115/231] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:00 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:00.751011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937baefb-3766-4262-81d6-0a414217de7b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:00.805610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a1870-a2ac-4860-9e74-1e1ae4d3deeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:00.806241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a1870-a2ac-4860-9e74-1e1ae4d3deeb None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:00.806516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a1870-a2ac-4860-9e74-1e1ae4d3deeb None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:00.806783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a1870-a2ac-4860-9e74-1e1ae4d3deeb None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:00.809041 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a1870-a2ac-4860-9e74-1e1ae4d3deeb None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:00.810501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a1870-a2ac-4860-9e74-1e1ae4d3deeb None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:00.812183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 117/232] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:00 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:00.818803 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a1870-a2ac-4860-9e74-1e1ae4d3deeb None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:00.828911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:00.903778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937baefb-3766-4262-81d6-0a414217de7b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:00.934904 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78d2a2-6f1e-42fd-bdab-88e55ef74b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:00.936032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78d2a2-6f1e-42fd-bdab-88e55ef74b29 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:00.936469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78d2a2-6f1e-42fd-bdab-88e55ef74b29 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:00.936892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78d2a2-6f1e-42fd-bdab-88e55ef74b29 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7/users/10b61d42978349cfba564d042f7dcdb9/roles/9df5a7d7691c49b9a78f0db0a050b7fe` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:00.940068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78d2a2-6f1e-42fd-bdab-88e55ef74b29 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9ba9949b27349ab83f686eee37fe6f7, user_id=10b61d42978349cfba564d042f7dcdb9, role_id=9df5a7d7691c49b9a78f0db0a050b7fe)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:00.942175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78d2a2-6f1e-42fd-bdab-88e55ef74b29 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:00.965251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 116/233] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:38:00 2022] PUT /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7/users/10b61d42978349cfba564d042f7dcdb9/roles/9df5a7d7691c49b9a78f0db0a050b7fe => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:01.000614 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a1870-a2ac-4860-9e74-1e1ae4d3deeb None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:01.039384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6a622-cfe3-4342-9599-90925026f83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:01.040232 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6a622-cfe3-4342-9599-90925026f83b None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:01.040509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6a622-cfe3-4342-9599-90925026f83b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:01.040836 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6a622-cfe3-4342-9599-90925026f83b None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7/users/10b61d42978349cfba564d042f7dcdb9/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:01.042901 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6a622-cfe3-4342-9599-90925026f83b None admin] RBAC: Authorizing `identity:create_grant(project_id=c9ba9949b27349ab83f686eee37fe6f7, user_id=10b61d42978349cfba564d042f7dcdb9, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:01.044536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6a622-cfe3-4342-9599-90925026f83b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:01.068978 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 118/234] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:38:00 2022] PUT /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7/users/10b61d42978349cfba564d042f7dcdb9/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:01.260885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1dc10-95cf-43c3-8c75-db0fbf9a077c None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:01.261373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1dc10-95cf-43c3-8c75-db0fbf9a077c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:01.261373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1dc10-95cf-43c3-8c75-db0fbf9a077c None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:01.262980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 117/235] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:01.268454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abe4c7-f9b8-4655-8ca3-f8a9342e82ac None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:01.268746 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abe4c7-f9b8-4655-8ca3-f8a9342e82ac None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:01.269015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abe4c7-f9b8-4655-8ca3-f8a9342e82ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:01.284924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6822-1eac-4c5c-aa88-d12dabd3bf29 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:01.285231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6822-1eac-4c5c-aa88-d12dabd3bf29 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:01.285481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6822-1eac-4c5c-aa88-d12dabd3bf29 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:01.286808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 118/236] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:38:01.288667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-93abe4c7-f9b8-4655-8ca3-f8a9342e82ac None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:01.289867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-93abe4c7-f9b8-4655-8ca3-f8a9342e82ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:01.292132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b32eb-e1d2-446a-81a0-cd3d0cba6302 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:01.292412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b32eb-e1d2-446a-81a0-cd3d0cba6302 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:01.292686 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b32eb-e1d2-446a-81a0-cd3d0cba6302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:01.319777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-238b32eb-e1d2-446a-81a0-cd3d0cba6302 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:01.320987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-238b32eb-e1d2-446a-81a0-cd3d0cba6302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:01.404154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:01.418354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 119/237] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:01.422594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:01.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 119/238] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:01.800818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abe4c7-f9b8-4655-8ca3-f8a9342e82ac None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:01.824701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b32eb-e1d2-446a-81a0-cd3d0cba6302 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:01.990016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abe4c7-f9b8-4655-8ca3-f8a9342e82ac None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:02.010503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b32eb-e1d2-446a-81a0-cd3d0cba6302 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:02.031522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e62544-8b01-46c9-a33e-7c6a612a8514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:02.032469 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e62544-8b01-46c9-a33e-7c6a612a8514 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:02.032947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e62544-8b01-46c9-a33e-7c6a612a8514 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:02.033417 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e62544-8b01-46c9-a33e-7c6a612a8514 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:02.034752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e62544-8b01-46c9-a33e-7c6a612a8514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:02.036454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e62544-8b01-46c9-a33e-7c6a612a8514 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:02.038218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 120/239] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:02.039597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afd900-c005-47a8-b092-d45eeb5abee0 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:02.040099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afd900-c005-47a8-b092-d45eeb5abee0 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:02.040568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afd900-c005-47a8-b092-d45eeb5abee0 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:02.046021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 121/240] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:02.046974 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afd900-c005-47a8-b092-d45eeb5abee0 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:02.050117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b519568-198f-4a13-8f0d-8cd40036abde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:02.050962 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b519568-198f-4a13-8f0d-8cd40036abde None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:02.051185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b519568-198f-4a13-8f0d-8cd40036abde None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:02.051460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b519568-198f-4a13-8f0d-8cd40036abde None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:02.065211 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b519568-198f-4a13-8f0d-8cd40036abde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-8b519568-198f-4a13-8f0d-8cd40036abde None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:02.070670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:02.071541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:02.072379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 120/241] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:02.073503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56045990-390c-4c8d-8596-e90823156e51 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:02.073660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56045990-390c-4c8d-8596-e90823156e51 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:02.073926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56045990-390c-4c8d-8596-e90823156e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:02.097123 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-56045990-390c-4c8d-8596-e90823156e51 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:02.098686 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-56045990-390c-4c8d-8596-e90823156e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:02.120449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:02.145475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:02.165048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afd900-c005-47a8-b092-d45eeb5abee0 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:02.191714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 121/242] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:02.192961 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56045990-390c-4c8d-8596-e90823156e51 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:02.208910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd4133-217e-4c7e-a54d-a9c362634a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:02.208910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd4133-217e-4c7e-a54d-a9c362634a6c None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:02.208910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd4133-217e-4c7e-a54d-a9c362634a6c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:02.208910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd4133-217e-4c7e-a54d-a9c362634a6c None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:02.209369 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd4133-217e-4c7e-a54d-a9c362634a6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:02.211983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd4133-217e-4c7e-a54d-a9c362634a6c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:02.248510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 122/243] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:38:02 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:38:02.370342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56045990-390c-4c8d-8596-e90823156e51 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:02.404845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df67af7-f068-4841-9d17-971fb4948cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:02.405471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df67af7-f068-4841-9d17-971fb4948cc6 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:02.405744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df67af7-f068-4841-9d17-971fb4948cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:02.406011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df67af7-f068-4841-9d17-971fb4948cc6 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:02.406687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df67af7-f068-4841-9d17-971fb4948cc6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:02.406989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df67af7-f068-4841-9d17-971fb4948cc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:02.409007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df67af7-f068-4841-9d17-971fb4948cc6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:02.421863 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 122/244] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:02.428419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd4133-217e-4c7e-a54d-a9c362634a6c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:02.542902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd4133-217e-4c7e-a54d-a9c362634a6c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:02.560504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:02.583886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d96f52-bce4-4e8e-8c35-edd5bd657ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:02.585634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d96f52-bce4-4e8e-8c35-edd5bd657ac2 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:02.586168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d96f52-bce4-4e8e-8c35-edd5bd657ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:02.586644 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d96f52-bce4-4e8e-8c35-edd5bd657ac2 None admin] PATH_INFO: `/v3/users/9f2be03176e24bd49439d5c12f122d47` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:02.604790 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d96f52-bce4-4e8e-8c35-edd5bd657ac2 None admin] RBAC: Authorizing `identity:get_user(user_id=9f2be03176e24bd49439d5c12f122d47)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:02.604965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df67af7-f068-4841-9d17-971fb4948cc6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:02.606701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d96f52-bce4-4e8e-8c35-edd5bd657ac2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:02.608646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 123/245] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:02 2022] GET /identity/v3/users/9f2be03176e24bd49439d5c12f122d47 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:02.614352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d96f52-bce4-4e8e-8c35-edd5bd657ac2 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:02.654832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:02.745180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df67af7-f068-4841-9d17-971fb4948cc6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:02.763040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d96f52-bce4-4e8e-8c35-edd5bd657ac2 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:02.802704 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa6ad7-2db0-4f0e-9f6f-a90f4f9daa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:02.803842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa6ad7-2db0-4f0e-9f6f-a90f4f9daa7b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:02.804274 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa6ad7-2db0-4f0e-9f6f-a90f4f9daa7b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:02.804555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa6ad7-2db0-4f0e-9f6f-a90f4f9daa7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:02.811827 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ee714-a285-4116-9120-53ad0e78ad80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:02.812429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ee714-a285-4116-9120-53ad0e78ad80 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:02.812748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ee714-a285-4116-9120-53ad0e78ad80 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:02.813024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ee714-a285-4116-9120-53ad0e78ad80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:02.818201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa6ad7-2db0-4f0e-9f6f-a90f4f9daa7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-77fa6ad7-2db0-4f0e-9f6f-a90f4f9daa7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:02.825030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:02.825981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:02.827257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 123/246] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:02.832342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa6ad7-2db0-4f0e-9f6f-a90f4f9daa7b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:02.880497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:02.909214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ee714-a285-4116-9120-53ad0e78ad80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:02.910604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ee714-a285-4116-9120-53ad0e78ad80 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:02.973642 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:02.982373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa6ad7-2db0-4f0e-9f6f-a90f4f9daa7b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:03.027093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0be1a6-33cf-4de2-a70c-f96a70363e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:03.027879 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0be1a6-33cf-4de2-a70c-f96a70363e4f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:03.028101 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0be1a6-33cf-4de2-a70c-f96a70363e4f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:03.028377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0be1a6-33cf-4de2-a70c-f96a70363e4f None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:03.029054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0be1a6-33cf-4de2-a70c-f96a70363e4f None admin] RBAC: Adding query filter params (name=member) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:03.029356 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0be1a6-33cf-4de2-a70c-f96a70363e4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:03.030953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0be1a6-33cf-4de2-a70c-f96a70363e4f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:03.034640 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 124/247] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 14:38:03.039848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 124/248] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:03.044186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ee714-a285-4116-9120-53ad0e78ad80 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:03.046867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0be1a6-33cf-4de2-a70c-f96a70363e4f None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:03.184511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:03.227171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ee714-a285-4116-9120-53ad0e78ad80 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:03.230459 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0be1a6-33cf-4de2-a70c-f96a70363e4f None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:03.286291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069fdb5-d70c-493c-a9b8-c2158c54a936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:03.286936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069fdb5-d70c-493c-a9b8-c2158c54a936 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:03.287418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069fdb5-d70c-493c-a9b8-c2158c54a936 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:03.287522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069fdb5-d70c-493c-a9b8-c2158c54a936 None admin] PATH_INFO: `/v3/users/ea39ac4cd0c644e79a6328aa5ae3e4d4` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:03.288646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069fdb5-d70c-493c-a9b8-c2158c54a936 None admin] RBAC: Authorizing `identity:get_user(user_id=ea39ac4cd0c644e79a6328aa5ae3e4d4)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:03.290110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069fdb5-d70c-493c-a9b8-c2158c54a936 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:03.291996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 125/249] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:03 2022] GET /identity/v3/users/ea39ac4cd0c644e79a6328aa5ae3e4d4 => generated 297 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:03.293224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169be79a-7914-4bad-83ac-7778faaf7173 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:03.293508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169be79a-7914-4bad-83ac-7778faaf7173 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:03.293780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169be79a-7914-4bad-83ac-7778faaf7173 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:03.295729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 126/250] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:03.296639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e161e7-d7af-4e8f-b5a9-684816b3ea57 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:03.296639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e161e7-d7af-4e8f-b5a9-684816b3ea57 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:03.299615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e161e7-d7af-4e8f-b5a9-684816b3ea57 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:03.299615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 127/251] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:03.300049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e161e7-d7af-4e8f-b5a9-684816b3ea57 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:03.306781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6acc9-e7c0-4ea0-ab1c-089c13561d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:03.307751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6acc9-e7c0-4ea0-ab1c-089c13561d2e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:03.308225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6acc9-e7c0-4ea0-ab1c-089c13561d2e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:03.308687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6acc9-e7c0-4ea0-ab1c-089c13561d2e None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:03.310055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b6acc9-e7c0-4ea0-ab1c-089c13561d2e None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:03.312270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b6acc9-e7c0-4ea0-ab1c-089c13561d2e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:03.314866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 125/252] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:03 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:03.316931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61593630-c5bb-4148-ad40-179fb313cc2f None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:03.317429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61593630-c5bb-4148-ad40-179fb313cc2f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:03.318062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61593630-c5bb-4148-ad40-179fb313cc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:03.329385 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:03.339855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-61593630-c5bb-4148-ad40-179fb313cc2f None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:03.341362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-61593630-c5bb-4148-ad40-179fb313cc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:03.421987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e161e7-d7af-4e8f-b5a9-684816b3ea57 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:03.436272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 126/253] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:03.436272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df964c82-e1c2-445f-a008-7acaf99f2cec None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:03.436272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df964c82-e1c2-445f-a008-7acaf99f2cec None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:03.436272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df964c82-e1c2-445f-a008-7acaf99f2cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:03.469145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-df964c82-e1c2-445f-a008-7acaf99f2cec None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:03.470375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-df964c82-e1c2-445f-a008-7acaf99f2cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:03.483132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a735d-1e37-42a4-a275-ed0e6c65cca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:03.483889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a735d-1e37-42a4-a275-ed0e6c65cca8 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:03.484170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a735d-1e37-42a4-a275-ed0e6c65cca8 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:03.484441 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a735d-1e37-42a4-a275-ed0e6c65cca8 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:03.486106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a735d-1e37-42a4-a275-ed0e6c65cca8 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:03.488487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a735d-1e37-42a4-a275-ed0e6c65cca8 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:03.490117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 128/254] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:03 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:03.490881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a735d-1e37-42a4-a275-ed0e6c65cca8 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:03.589762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 127/255] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:03.590675 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df964c82-e1c2-445f-a008-7acaf99f2cec None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:03.601332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:03.687724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:03.709921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a735d-1e37-42a4-a275-ed0e6c65cca8 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:03.796187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df964c82-e1c2-445f-a008-7acaf99f2cec None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:03.804382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03f9a1-01d8-45c6-bb82-be7c52688522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:03.804992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03f9a1-01d8-45c6-bb82-be7c52688522 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:03.805269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03f9a1-01d8-45c6-bb82-be7c52688522 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:03.805633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03f9a1-01d8-45c6-bb82-be7c52688522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:03.806271 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03f9a1-01d8-45c6-bb82-be7c52688522 None admin] RBAC: Adding query filter params (role.id=a853ef2aaba2459f86b50cae95d23577, scope.project.id=c9ba9949b27349ab83f686eee37fe6f7, user.id=9f2be03176e24bd49439d5c12f122d47) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:03.806580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03f9a1-01d8-45c6-bb82-be7c52688522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:03.808105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03f9a1-01d8-45c6-bb82-be7c52688522 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:03.821569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 129/256] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:03 2022] GET /identity/v3/role_assignments?user.id=9f2be03176e24bd49439d5c12f122d47&scope.project.id=c9ba9949b27349ab83f686eee37fe6f7&role.id=a853ef2aaba2459f86b50cae95d23577 => generated 259 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:03.823165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03f9a1-01d8-45c6-bb82-be7c52688522 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:03.846546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e49971-d515-4039-9231-e3a4aa4e757e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:03.847250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e49971-d515-4039-9231-e3a4aa4e757e None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:03.847482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e49971-d515-4039-9231-e3a4aa4e757e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:03.847772 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e49971-d515-4039-9231-e3a4aa4e757e None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7/users/ea39ac4cd0c644e79a6328aa5ae3e4d4/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:03.850446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e49971-d515-4039-9231-e3a4aa4e757e None admin] RBAC: Authorizing `identity:create_grant(project_id=c9ba9949b27349ab83f686eee37fe6f7, user_id=ea39ac4cd0c644e79a6328aa5ae3e4d4, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:03.851958 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e49971-d515-4039-9231-e3a4aa4e757e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:03.874121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 128/257] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:38:03 2022] PUT /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7/users/ea39ac4cd0c644e79a6328aa5ae3e4d4/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:03.991774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03f9a1-01d8-45c6-bb82-be7c52688522 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:04.042469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473cc8a-31c3-4ee3-b7da-55d90f9a4320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:04.043136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473cc8a-31c3-4ee3-b7da-55d90f9a4320 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:04.043419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473cc8a-31c3-4ee3-b7da-55d90f9a4320 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:04.043705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473cc8a-31c3-4ee3-b7da-55d90f9a4320 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:04.061795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473cc8a-31c3-4ee3-b7da-55d90f9a4320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-1473cc8a-31c3-4ee3-b7da-55d90f9a4320 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:04.067854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:04.068807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:04.070020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 130/258] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:38:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:04.077028 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e49971-d515-4039-9231-e3a4aa4e757e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:04.103583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473cc8a-31c3-4ee3-b7da-55d90f9a4320 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:04.126670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:04.201535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:04.220876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473cc8a-31c3-4ee3-b7da-55d90f9a4320 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:04.224924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e49971-d515-4039-9231-e3a4aa4e757e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:04.261827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a7dca-30c4-4a41-9e2c-90c187b197c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:04.262392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a7dca-30c4-4a41-9e2c-90c187b197c7 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:04.262672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a7dca-30c4-4a41-9e2c-90c187b197c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:04.262949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a7dca-30c4-4a41-9e2c-90c187b197c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:04.292435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d65a5-8954-4f9e-bd15-9be3b662e459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:04.293055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d65a5-8954-4f9e-bd15-9be3b662e459 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:04.293338 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d65a5-8954-4f9e-bd15-9be3b662e459 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:04.293615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d65a5-8954-4f9e-bd15-9be3b662e459 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:04.294300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d65a5-8954-4f9e-bd15-9be3b662e459 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:04.294602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d65a5-8954-4f9e-bd15-9be3b662e459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:04.296024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d65a5-8954-4f9e-bd15-9be3b662e459 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:04.298849 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a7dca-30c4-4a41-9e2c-90c187b197c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-409a7dca-30c4-4a41-9e2c-90c187b197c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:04.304323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:04.306705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:04.308712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 131/259] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:38:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:04.315276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a7dca-30c4-4a41-9e2c-90c187b197c7 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:04.320684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 129/260] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:04.326578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d65a5-8954-4f9e-bd15-9be3b662e459 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:04.472855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:04.512038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a7dca-30c4-4a41-9e2c-90c187b197c7 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:04.516021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d65a5-8954-4f9e-bd15-9be3b662e459 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:04.568060 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:04.578462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ab5e5-aa4f-4778-b8b1-8670bef443c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:04.579703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ab5e5-aa4f-4778-b8b1-8670bef443c6 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:04.580114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ab5e5-aa4f-4778-b8b1-8670bef443c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:04.580394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ab5e5-aa4f-4778-b8b1-8670bef443c6 None admin] PATH_INFO: `/v3/users/10b61d42978349cfba564d042f7dcdb9` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:04.582518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ab5e5-aa4f-4778-b8b1-8670bef443c6 None admin] RBAC: Authorizing `identity:get_user(user_id=10b61d42978349cfba564d042f7dcdb9)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:04.584045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ab5e5-aa4f-4778-b8b1-8670bef443c6 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:04.586560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 130/261] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:04 2022] GET /identity/v3/users/10b61d42978349cfba564d042f7dcdb9 => generated 283 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:04.591638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ab5e5-aa4f-4778-b8b1-8670bef443c6 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:04.628394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02570c9b-d12a-4667-92e0-a97bae7fdb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:04.629028 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570c9b-d12a-4667-92e0-a97bae7fdb2c None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:04.629316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570c9b-d12a-4667-92e0-a97bae7fdb2c None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:04.629593 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570c9b-d12a-4667-92e0-a97bae7fdb2c None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:04.630273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02570c9b-d12a-4667-92e0-a97bae7fdb2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:04.630579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02570c9b-d12a-4667-92e0-a97bae7fdb2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:04.632010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02570c9b-d12a-4667-92e0-a97bae7fdb2c None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:04.646121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 132/262] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:04.653382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02570c9b-d12a-4667-92e0-a97bae7fdb2c None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:04.742577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ab5e5-aa4f-4778-b8b1-8670bef443c6 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:04.798766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277d2d4-8ef6-4b8c-a8d5-fe20e6319950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:04.799304 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277d2d4-8ef6-4b8c-a8d5-fe20e6319950 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:04.799584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277d2d4-8ef6-4b8c-a8d5-fe20e6319950 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:04.799883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277d2d4-8ef6-4b8c-a8d5-fe20e6319950 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:04.905515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277d2d4-8ef6-4b8c-a8d5-fe20e6319950 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:04.905981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:04.907613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277d2d4-8ef6-4b8c-a8d5-fe20e6319950 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:04.917603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02570c9b-d12a-4667-92e0-a97bae7fdb2c None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:05.021023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 131/263] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 14:38:05.026621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277d2d4-8ef6-4b8c-a8d5-fe20e6319950 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:05.044938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126328d-fa1e-4bd3-863b-e45eca7c4676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:05.045595 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126328d-fa1e-4bd3-863b-e45eca7c4676 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:05.045877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126328d-fa1e-4bd3-863b-e45eca7c4676 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:05.046155 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126328d-fa1e-4bd3-863b-e45eca7c4676 None admin] PATH_INFO: `/v3/users/10b61d42978349cfba564d042f7dcdb9` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:05.047316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126328d-fa1e-4bd3-863b-e45eca7c4676 None admin] RBAC: Authorizing `identity:get_user(user_id=10b61d42978349cfba564d042f7dcdb9)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:05.049528 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126328d-fa1e-4bd3-863b-e45eca7c4676 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:05.051376 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 133/264] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:04 2022] GET /identity/v3/users/10b61d42978349cfba564d042f7dcdb9 => generated 283 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:05.060601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126328d-fa1e-4bd3-863b-e45eca7c4676 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:05.076452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:05.128768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277d2d4-8ef6-4b8c-a8d5-fe20e6319950 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:05.144103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:05.172221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef4216-dd4c-4d01-8d6b-a2edd45fa604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:05.173458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef4216-dd4c-4d01-8d6b-a2edd45fa604 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:05.173741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef4216-dd4c-4d01-8d6b-a2edd45fa604 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:05.174009 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef4216-dd4c-4d01-8d6b-a2edd45fa604 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:05.177603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef4216-dd4c-4d01-8d6b-a2edd45fa604 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:05.179149 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef4216-dd4c-4d01-8d6b-a2edd45fa604 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:05.180854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 132/265] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:05 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:05.187439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef4216-dd4c-4d01-8d6b-a2edd45fa604 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:05.298446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef4216-dd4c-4d01-8d6b-a2edd45fa604 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:05.339358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ef6ad-8654-4a05-9c94-06a1a914a434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:05.340286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ef6ad-8654-4a05-9c94-06a1a914a434 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:05.340702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ef6ad-8654-4a05-9c94-06a1a914a434 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:05.343906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ef6ad-8654-4a05-9c94-06a1a914a434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:05.343906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ef6ad-8654-4a05-9c94-06a1a914a434 None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=c9ba9949b27349ab83f686eee37fe6f7, user.id=10b61d42978349cfba564d042f7dcdb9) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:05.343906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ef6ad-8654-4a05-9c94-06a1a914a434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:05.343906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ef6ad-8654-4a05-9c94-06a1a914a434 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:05.351586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 133/266] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:05 2022] GET /identity/v3/role_assignments?user.id=10b61d42978349cfba564d042f7dcdb9&scope.project.id=c9ba9949b27349ab83f686eee37fe6f7&role.id=9a439cf8eef34ada85428b378878e63e => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:05.358824 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e37c9-605b-48b8-a39c-3a5c4ad791d4 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:05.359715 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e37c9-605b-48b8-a39c-3a5c4ad791d4 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:05.360164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e37c9-605b-48b8-a39c-3a5c4ad791d4 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:05.362606 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 134/267] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:05 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:05.376153 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c545b-3ff4-4564-836d-04c5920d2a78 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:05.376717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c545b-3ff4-4564-836d-04c5920d2a78 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:05.377127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c545b-3ff4-4564-836d-04c5920d2a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:05.408821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-ca1c545b-3ff4-4564-836d-04c5920d2a78 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:05.409939 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126328d-fa1e-4bd3-863b-e45eca7c4676 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:05.410458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ca1c545b-3ff4-4564-836d-04c5920d2a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:05.444131 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:05.496635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 135/268] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:05.537564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b250bff-c8d7-4dba-88f1-8df5c22482e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:05.538202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b250bff-c8d7-4dba-88f1-8df5c22482e3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:05.538652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b250bff-c8d7-4dba-88f1-8df5c22482e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:05.539088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b250bff-c8d7-4dba-88f1-8df5c22482e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:05.642530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b250bff-c8d7-4dba-88f1-8df5c22482e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:05.644281 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b250bff-c8d7-4dba-88f1-8df5c22482e3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:05.659592 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:05.777709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 134/269] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:05.783917 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c545b-3ff4-4564-836d-04c5920d2a78 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:05.800561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43478f54-e459-4725-a17c-2f8620c29c32 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:05.800835 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43478f54-e459-4725-a17c-2f8620c29c32 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:05.801110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43478f54-e459-4725-a17c-2f8620c29c32 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:05.802712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 135/270] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:38:05.812168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b0a9d-fcc3-43a4-ab48-42af8e2ec708 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:05.812651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b0a9d-fcc3-43a4-ab48-42af8e2ec708 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:05.813078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b0a9d-fcc3-43a4-ab48-42af8e2ec708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:05.839374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-6b3b0a9d-fcc3-43a4-ab48-42af8e2ec708 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:05.842062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6b3b0a9d-fcc3-43a4-ab48-42af8e2ec708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:05.967712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 136/271] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:05.980520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b0a9d-fcc3-43a4-ab48-42af8e2ec708 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:06.000804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c545b-3ff4-4564-836d-04c5920d2a78 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:06.019817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:06.061300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da492d-d4c8-41fc-b614-cfc511055380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:06.061961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da492d-d4c8-41fc-b614-cfc511055380 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.062234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da492d-d4c8-41fc-b614-cfc511055380 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.062502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da492d-d4c8-41fc-b614-cfc511055380 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:06.063666 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da492d-d4c8-41fc-b614-cfc511055380 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:06.069184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da492d-d4c8-41fc-b614-cfc511055380 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:06.071027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 136/272] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:05 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:06.076896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da492d-d4c8-41fc-b614-cfc511055380 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:06.099505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b0a9d-fcc3-43a4-ab48-42af8e2ec708 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:06.113147 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:06.144975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85b8f3-0f72-4a9a-94d7-f98ce45e6e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:06.145903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85b8f3-0f72-4a9a-94d7-f98ce45e6e7d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.146353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85b8f3-0f72-4a9a-94d7-f98ce45e6e7d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.148262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85b8f3-0f72-4a9a-94d7-f98ce45e6e7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:06.170580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85b8f3-0f72-4a9a-94d7-f98ce45e6e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-9e85b8f3-0f72-4a9a-94d7-f98ce45e6e7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:06.183324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:06.184288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:06.185110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:06.185110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:06.185110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:06.185110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:06.185110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:06.185110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:06.185110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:06.185110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:06.185110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:06.185110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:06.185110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:06.185110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:06.191578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 137/273] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:06.194573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85b8f3-0f72-4a9a-94d7-f98ce45e6e7d None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:06.293558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da492d-d4c8-41fc-b614-cfc511055380 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:06.305926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:06.312676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85b8f3-0f72-4a9a-94d7-f98ce45e6e7d None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:06.352515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d65599-db6a-474e-a120-4f384458f32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:06.353520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d65599-db6a-474e-a120-4f384458f32a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.353936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d65599-db6a-474e-a120-4f384458f32a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.354366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d65599-db6a-474e-a120-4f384458f32a None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:06.355165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d65599-db6a-474e-a120-4f384458f32a None admin] RBAC: Adding query filter params (name=member) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:06.355606 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d65599-db6a-474e-a120-4f384458f32a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:06.357428 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d65599-db6a-474e-a120-4f384458f32a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:06.367334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 138/274] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:06.368926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb701d7-26b6-44ac-a93e-d2525f477959 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.369405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb701d7-26b6-44ac-a93e-d2525f477959 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.369608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334ec16-63d8-402b-868a-265b952b42fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:06.370103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb701d7-26b6-44ac-a93e-d2525f477959 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:06.370205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334ec16-63d8-402b-868a-265b952b42fb None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.370461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334ec16-63d8-402b-868a-265b952b42fb None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.370740 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334ec16-63d8-402b-868a-265b952b42fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:06.371477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334ec16-63d8-402b-868a-265b952b42fb None admin] RBAC: Adding query filter params (role.id=9df5a7d7691c49b9a78f0db0a050b7fe, scope.project.id=c9ba9949b27349ab83f686eee37fe6f7, user.id=10b61d42978349cfba564d042f7dcdb9) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:06.371827 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334ec16-63d8-402b-868a-265b952b42fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:06.372494 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 139/275] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:06.373537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334ec16-63d8-402b-868a-265b952b42fb None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:06.384007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb701d7-26b6-44ac-a93e-d2525f477959 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:06.384631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 137/276] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:06 2022] GET /identity/v3/role_assignments?user.id=10b61d42978349cfba564d042f7dcdb9&scope.project.id=c9ba9949b27349ab83f686eee37fe6f7&role.id=9df5a7d7691c49b9a78f0db0a050b7fe => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:06.385856 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2ce18-15df-45e7-b2e5-648f93fd2a7a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.386141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2ce18-15df-45e7-b2e5-648f93fd2a7a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.386417 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2ce18-15df-45e7-b2e5-648f93fd2a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:06.414822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-13a2ce18-15df-45e7-b2e5-648f93fd2a7a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:06.416180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-13a2ce18-15df-45e7-b2e5-648f93fd2a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:06.515863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 138/277] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:06.517498 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2ce18-15df-45e7-b2e5-648f93fd2a7a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:06.533112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb701d7-26b6-44ac-a93e-d2525f477959 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:06.541863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:06.577765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c02bd6-9fd4-4b9c-bde8-02a0c2199277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:06.579215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c02bd6-9fd4-4b9c-bde8-02a0c2199277 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.580044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c02bd6-9fd4-4b9c-bde8-02a0c2199277 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.580044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c02bd6-9fd4-4b9c-bde8-02a0c2199277 None admin] PATH_INFO: `/v3/users/297d5031504e4dfeb6f76be284f59de4` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:06.596418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c02bd6-9fd4-4b9c-bde8-02a0c2199277 None admin] RBAC: Authorizing `identity:get_user(user_id=297d5031504e4dfeb6f76be284f59de4)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:06.597884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c02bd6-9fd4-4b9c-bde8-02a0c2199277 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:06.599720 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 140/278] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:06 2022] GET /identity/v3/users/297d5031504e4dfeb6f76be284f59de4 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:06.600723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58141f5e-df4c-4c40-96ec-30c7021e0b69 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.601003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58141f5e-df4c-4c40-96ec-30c7021e0b69 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.601272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58141f5e-df4c-4c40-96ec-30c7021e0b69 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:06.603807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 141/279] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:06.605142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58141f5e-df4c-4c40-96ec-30c7021e0b69 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:06.625386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:06.646451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2ce18-15df-45e7-b2e5-648f93fd2a7a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:06.692364 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71403a68-027d-4af9-8b5d-82e2df9fd0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:06.693010 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71403a68-027d-4af9-8b5d-82e2df9fd0ac None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.693290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71403a68-027d-4af9-8b5d-82e2df9fd0ac None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.693564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71403a68-027d-4af9-8b5d-82e2df9fd0ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:06.694685 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71403a68-027d-4af9-8b5d-82e2df9fd0ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:06.696180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71403a68-027d-4af9-8b5d-82e2df9fd0ac None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:06.697908 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 139/280] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:06.703238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9080c8e-1674-4fce-9557-f6acbb3b718e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.703525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9080c8e-1674-4fce-9557-f6acbb3b718e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.703825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9080c8e-1674-4fce-9557-f6acbb3b718e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:06.723174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-f9080c8e-1674-4fce-9557-f6acbb3b718e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:06.724559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f9080c8e-1674-4fce-9557-f6acbb3b718e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:06.735946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58141f5e-df4c-4c40-96ec-30c7021e0b69 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:06.770482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3225e-5574-4064-af77-8891f67186e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:06.771020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3225e-5574-4064-af77-8891f67186e8 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.771302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3225e-5574-4064-af77-8891f67186e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.771582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3225e-5574-4064-af77-8891f67186e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:06.772342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3225e-5574-4064-af77-8891f67186e8 None admin] RBAC: Adding query filter params (role.id=603573d2738b49c389ad913208d2ced1, user.id=297d5031504e4dfeb6f76be284f59de4, scope.system=all) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:06.772653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3225e-5574-4064-af77-8891f67186e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:06.774115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3225e-5574-4064-af77-8891f67186e8 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:06.779210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 142/281] 10.222.0.36 () {56 vars in 1308 bytes} [Thu Aug 18 14:38:06 2022] GET /identity/v3/role_assignments?user.id=297d5031504e4dfeb6f76be284f59de4&scope.system=all&role.id=603573d2738b49c389ad913208d2ced1 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:06.779917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3225e-5574-4064-af77-8891f67186e8 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:06.805541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 140/282] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:06.806737 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9080c8e-1674-4fce-9557-f6acbb3b718e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:06.833429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:06.903118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3225e-5574-4064-af77-8891f67186e8 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:06.911439 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:06.948392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f87a-cad2-4fae-a3d2-a7302c836841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:06.949113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f87a-cad2-4fae-a3d2-a7302c836841 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.949456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f87a-cad2-4fae-a3d2-a7302c836841 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.949649 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9080c8e-1674-4fce-9557-f6acbb3b718e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:06.949866 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f87a-cad2-4fae-a3d2-a7302c836841 None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:06.950595 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6f87a-cad2-4fae-a3d2-a7302c836841 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:06.952251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6f87a-cad2-4fae-a3d2-a7302c836841 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:06.978486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 143/283] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:38:06 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:38:06.987547 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d60f9f-c910-4b67-8dbd-2137f4f666b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:06.988244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d60f9f-c910-4b67-8dbd-2137f4f666b6 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:06.988525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d60f9f-c910-4b67-8dbd-2137f4f666b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:06.988797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d60f9f-c910-4b67-8dbd-2137f4f666b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:07.002919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d60f9f-c910-4b67-8dbd-2137f4f666b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-61d60f9f-c910-4b67-8dbd-2137f4f666b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:07.008150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:07.009104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:07.009906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 141/284] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:07.015385 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f87a-cad2-4fae-a3d2-a7302c836841 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:07.092354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d60f9f-c910-4b67-8dbd-2137f4f666b6 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:07.128965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f87a-cad2-4fae-a3d2-a7302c836841 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:07.167657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fb27a-7bf7-4112-bdc9-83d26c36758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:07.168309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fb27a-7bf7-4112-bdc9-83d26c36758a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:07.168589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fb27a-7bf7-4112-bdc9-83d26c36758a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:07.168861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fb27a-7bf7-4112-bdc9-83d26c36758a None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:07.170171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fb27a-7bf7-4112-bdc9-83d26c36758a None admin] RBAC: Adding query filter params (name=reader) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:07.170171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fb27a-7bf7-4112-bdc9-83d26c36758a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:07.171332 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fb27a-7bf7-4112-bdc9-83d26c36758a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:07.174742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:07.189792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 144/285] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:07.191280 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fb27a-7bf7-4112-bdc9-83d26c36758a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:07.203218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:07.233007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d60f9f-c910-4b67-8dbd-2137f4f666b6 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:07.277728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f459da4-caa1-4c68-8e0d-8054a9e64e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:07.278706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f459da4-caa1-4c68-8e0d-8054a9e64e95 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:07.279212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f459da4-caa1-4c68-8e0d-8054a9e64e95 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:07.279715 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f459da4-caa1-4c68-8e0d-8054a9e64e95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:07.292013 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f459da4-caa1-4c68-8e0d-8054a9e64e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-8f459da4-caa1-4c68-8e0d-8054a9e64e95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:07.296753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:07.297720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:07.299370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 142/286] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:07.303992 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f459da4-caa1-4c68-8e0d-8054a9e64e95 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:07.318359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fb27a-7bf7-4112-bdc9-83d26c36758a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:07.360907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803f2af-2d1d-4e81-aacc-787bdc55b502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:07.361468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803f2af-2d1d-4e81-aacc-787bdc55b502 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:07.361745 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803f2af-2d1d-4e81-aacc-787bdc55b502 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:07.362018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803f2af-2d1d-4e81-aacc-787bdc55b502 None admin] PATH_INFO: `/v3/users/9f2be03176e24bd49439d5c12f122d47` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:07.363915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803f2af-2d1d-4e81-aacc-787bdc55b502 None admin] RBAC: Authorizing `identity:get_user(user_id=9f2be03176e24bd49439d5c12f122d47)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:07.365368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803f2af-2d1d-4e81-aacc-787bdc55b502 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:07.367255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 145/287] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:07 2022] GET /identity/v3/users/9f2be03176e24bd49439d5c12f122d47 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:07.372680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803f2af-2d1d-4e81-aacc-787bdc55b502 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:07.436241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f459da4-caa1-4c68-8e0d-8054a9e64e95 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:07.456562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:07.483893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f998e-e051-4f73-a9e9-756c952c2679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:07.484498 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f998e-e051-4f73-a9e9-756c952c2679 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:07.484777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f998e-e051-4f73-a9e9-756c952c2679 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:07.485043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f998e-e051-4f73-a9e9-756c952c2679 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:07.485738 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f998e-e051-4f73-a9e9-756c952c2679 None admin] RBAC: Adding query filter params (name=member) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:07.486032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f998e-e051-4f73-a9e9-756c952c2679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:07.487409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f998e-e051-4f73-a9e9-756c952c2679 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:07.489784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:07.494615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803f2af-2d1d-4e81-aacc-787bdc55b502 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:07.499088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 143/288] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:07.504831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f998e-e051-4f73-a9e9-756c952c2679 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:07.534207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c31b8d-f9dc-4b61-a66d-89cde21a350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:07.534844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c31b8d-f9dc-4b61-a66d-89cde21a350b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:07.535123 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c31b8d-f9dc-4b61-a66d-89cde21a350b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:07.535393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c31b8d-f9dc-4b61-a66d-89cde21a350b None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:07.536677 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c31b8d-f9dc-4b61-a66d-89cde21a350b None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:07.540640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c31b8d-f9dc-4b61-a66d-89cde21a350b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:07.542198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 146/289] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:07 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:07.548392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c31b8d-f9dc-4b61-a66d-89cde21a350b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:07.612177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f998e-e051-4f73-a9e9-756c952c2679 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:07.647233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff5d9f-0295-4218-b550-35c981726f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:07.648327 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff5d9f-0295-4218-b550-35c981726f9a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:07.648780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff5d9f-0295-4218-b550-35c981726f9a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:07.649560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff5d9f-0295-4218-b550-35c981726f9a None admin] PATH_INFO: `/v3/users/ea39ac4cd0c644e79a6328aa5ae3e4d4` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:07.650876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff5d9f-0295-4218-b550-35c981726f9a None admin] RBAC: Authorizing `identity:get_user(user_id=ea39ac4cd0c644e79a6328aa5ae3e4d4)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:07.652518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff5d9f-0295-4218-b550-35c981726f9a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:07.654951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 144/290] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:07 2022] GET /identity/v3/users/ea39ac4cd0c644e79a6328aa5ae3e4d4 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:07.660637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff5d9f-0295-4218-b550-35c981726f9a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:07.745826 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c31b8d-f9dc-4b61-a66d-89cde21a350b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:07.764103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:07.786854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff5d9f-0295-4218-b550-35c981726f9a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:07.823956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e2e34-b619-4c90-8191-5e97e707790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:07.824914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e2e34-b619-4c90-8191-5e97e707790d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:07.825422 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e2e34-b619-4c90-8191-5e97e707790d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:07.825889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e2e34-b619-4c90-8191-5e97e707790d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:07.883626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ecb28-c741-46b7-a189-2c92f2fd91b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:07.884248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ecb28-c741-46b7-a189-2c92f2fd91b9 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:07.884524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ecb28-c741-46b7-a189-2c92f2fd91b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:07.884794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ecb28-c741-46b7-a189-2c92f2fd91b9 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7/users/9f2be03176e24bd49439d5c12f122d47/roles/a853ef2aaba2459f86b50cae95d23577` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:07.886707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ecb28-c741-46b7-a189-2c92f2fd91b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9ba9949b27349ab83f686eee37fe6f7, user_id=9f2be03176e24bd49439d5c12f122d47, role_id=a853ef2aaba2459f86b50cae95d23577)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:07.891783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ecb28-c741-46b7-a189-2c92f2fd91b9 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:07.895918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:07.900181 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e2e34-b619-4c90-8191-5e97e707790d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:07.901902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e2e34-b619-4c90-8191-5e97e707790d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:07.908285 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 147/291] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:38:07 2022] PUT /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7/users/9f2be03176e24bd49439d5c12f122d47/roles/a853ef2aaba2459f86b50cae95d23577 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:07.984637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:08.023282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 145/292] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 14:38:08.029511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ecb28-c741-46b7-a189-2c92f2fd91b9 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:08.173135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ecb28-c741-46b7-a189-2c92f2fd91b9 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:08.210309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71b592-be79-4fea-aca7-3709d0ae2d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:08.210943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71b592-be79-4fea-aca7-3709d0ae2d9b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:08.211218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71b592-be79-4fea-aca7-3709d0ae2d9b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:08.211489 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71b592-be79-4fea-aca7-3709d0ae2d9b None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:08.212665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71b592-be79-4fea-aca7-3709d0ae2d9b None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:08.214110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71b592-be79-4fea-aca7-3709d0ae2d9b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:08.216084 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 148/293] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:08 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:08.222528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e2e34-b619-4c90-8191-5e97e707790d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:08.340169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e2e34-b619-4c90-8191-5e97e707790d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:08.376461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcd052-18e3-4120-ad84-0c73568a6f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:08.377070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcd052-18e3-4120-ad84-0c73568a6f95 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:08.377346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcd052-18e3-4120-ad84-0c73568a6f95 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:08.377614 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcd052-18e3-4120-ad84-0c73568a6f95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:08.378342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcd052-18e3-4120-ad84-0c73568a6f95 None admin] RBAC: Adding query filter params (role.id=603573d2738b49c389ad913208d2ced1, scope.project.id=c9ba9949b27349ab83f686eee37fe6f7, user.id=ea39ac4cd0c644e79a6328aa5ae3e4d4) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:08.378651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcd052-18e3-4120-ad84-0c73568a6f95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:08.380145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcd052-18e3-4120-ad84-0c73568a6f95 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:08.386549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 146/294] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:08 2022] GET /identity/v3/role_assignments?user.id=ea39ac4cd0c644e79a6328aa5ae3e4d4&scope.project.id=c9ba9949b27349ab83f686eee37fe6f7&role.id=603573d2738b49c389ad913208d2ced1 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:08.987553 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5649f-222f-49f1-8623-b0caff59b487 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:08.988112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5649f-222f-49f1-8623-b0caff59b487 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:08.988112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5649f-222f-49f1-8623-b0caff59b487 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:08.989613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 149/295] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:08.995980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b6b59-8355-408c-863e-e49855bb76b8 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:08.996542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b6b59-8355-408c-863e-e49855bb76b8 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:08.997039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b6b59-8355-408c-863e-e49855bb76b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:09.015739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-414b6b59-8355-408c-863e-e49855bb76b8 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:09.017793 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-414b6b59-8355-408c-863e-e49855bb76b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:09.042867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:09.091804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 147/296] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:09.335969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efcd14-9514-4210-8b38-e858547cceca None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:09.336313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efcd14-9514-4210-8b38-e858547cceca None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:09.336513 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efcd14-9514-4210-8b38-e858547cceca None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:09.338111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 150/297] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:09.343227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565b1c5-1a9d-44ee-8f26-ec8660364040 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:09.343517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565b1c5-1a9d-44ee-8f26-ec8660364040 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:09.343829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565b1c5-1a9d-44ee-8f26-ec8660364040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:09.355158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efcd14-9514-4210-8b38-e858547cceca None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:09.364918 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-c565b1c5-1a9d-44ee-8f26-ec8660364040 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:09.366139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c565b1c5-1a9d-44ee-8f26-ec8660364040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:09.435636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:09.437727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 148/298] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:09.478524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efcd14-9514-4210-8b38-e858547cceca None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:09.512803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43844e-3081-4494-9493-4a2d006daf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:09.513434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43844e-3081-4494-9493-4a2d006daf0e None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:09.513706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43844e-3081-4494-9493-4a2d006daf0e None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:09.513974 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43844e-3081-4494-9493-4a2d006daf0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:09.524086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43844e-3081-4494-9493-4a2d006daf0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-ae43844e-3081-4494-9493-4a2d006daf0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:09.529198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:09.530109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:09.530895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:09.530895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:09.530895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:09.530895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:09.530895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:09.530895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:09.530895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:09.530895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:09.530895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:09.530895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:09.530895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:09.530895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:09.535090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 151/299] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:09.537783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565b1c5-1a9d-44ee-8f26-ec8660364040 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:09.635139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:09.647522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565b1c5-1a9d-44ee-8f26-ec8660364040 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:09.683561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c95ddc-dcda-46d5-8fdc-fea86b2f11ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:09.684202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c95ddc-dcda-46d5-8fdc-fea86b2f11ee None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:09.684483 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c95ddc-dcda-46d5-8fdc-fea86b2f11ee None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:09.684759 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c95ddc-dcda-46d5-8fdc-fea86b2f11ee None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:09.685427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c95ddc-dcda-46d5-8fdc-fea86b2f11ee None admin] RBAC: Adding query filter params (name=member) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:09.685730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c95ddc-dcda-46d5-8fdc-fea86b2f11ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:09.687121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c95ddc-dcda-46d5-8fdc-fea86b2f11ee None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:09.696786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 149/300] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:09.700649 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc33f31-eda2-4a6e-ba2f-d2c97b14fa72 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:09.700933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc33f31-eda2-4a6e-ba2f-d2c97b14fa72 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:09.701212 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc33f31-eda2-4a6e-ba2f-d2c97b14fa72 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:09.702945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c95ddc-dcda-46d5-8fdc-fea86b2f11ee None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:09.703105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 152/301] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:38:09.707474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a5f36-a014-42e5-a9c3-edbda8ce2382 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:09.707893 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a5f36-a014-42e5-a9c3-edbda8ce2382 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:09.709284 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a5f36-a014-42e5-a9c3-edbda8ce2382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:09.727862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-e10a5f36-a014-42e5-a9c3-edbda8ce2382 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:09.729151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e10a5f36-a014-42e5-a9c3-edbda8ce2382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:09.801244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 153/302] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:09.802264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a5f36-a014-42e5-a9c3-edbda8ce2382 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:09.820243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c95ddc-dcda-46d5-8fdc-fea86b2f11ee None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:09.856749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ea3ad-e3c6-4cc8-936d-f19b10a712dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:09.857384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ea3ad-e3c6-4cc8-936d-f19b10a712dd None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:09.857688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ea3ad-e3c6-4cc8-936d-f19b10a712dd None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:09.858161 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ea3ad-e3c6-4cc8-936d-f19b10a712dd None admin] PATH_INFO: `/v3/users/297d5031504e4dfeb6f76be284f59de4` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:09.859126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ea3ad-e3c6-4cc8-936d-f19b10a712dd None admin] RBAC: Authorizing `identity:get_user(user_id=297d5031504e4dfeb6f76be284f59de4)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:09.861189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ea3ad-e3c6-4cc8-936d-f19b10a712dd None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:09.863106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 150/303] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:09 2022] GET /identity/v3/users/297d5031504e4dfeb6f76be284f59de4 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:09.868313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ea3ad-e3c6-4cc8-936d-f19b10a712dd None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:09.869716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:09.902248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:09.944108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a5f36-a014-42e5-a9c3-edbda8ce2382 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:09.992228 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8354a5-e7d0-4f30-bd0c-b8f76844d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:09.993824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8354a5-e7d0-4f30-bd0c-b8f76844d5fb None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:09.994251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8354a5-e7d0-4f30-bd0c-b8f76844d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:09.994895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8354a5-e7d0-4f30-bd0c-b8f76844d5fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:10.005695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ea3ad-e3c6-4cc8-936d-f19b10a712dd None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:10.010305 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8354a5-e7d0-4f30-bd0c-b8f76844d5fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-1e8354a5-e7d0-4f30-bd0c-b8f76844d5fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:10.016996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:10.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:10.019752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:10.019752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:10.019752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:10.019752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:10.019752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:10.019752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:10.019752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:10.019752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:10.019752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:10.019752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:10.019752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:38:10.019752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:10.023732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 154/304] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:10.026839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8354a5-e7d0-4f30-bd0c-b8f76844d5fb None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:10.052923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac2ea3-ad2c-497a-a3c7-3444e051030d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:10.053604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac2ea3-ad2c-497a-a3c7-3444e051030d None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:10.053888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac2ea3-ad2c-497a-a3c7-3444e051030d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:10.054170 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac2ea3-ad2c-497a-a3c7-3444e051030d None admin] PATH_INFO: `/v3/system/users/297d5031504e4dfeb6f76be284f59de4/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:10.055752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac2ea3-ad2c-497a-a3c7-3444e051030d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=297d5031504e4dfeb6f76be284f59de4, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:10.057208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac2ea3-ad2c-497a-a3c7-3444e051030d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:10.068914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 151/305] 10.222.0.36 () {58 vars in 1334 bytes} [Thu Aug 18 14:38:09 2022] PUT /identity/v3/system/users/297d5031504e4dfeb6f76be284f59de4/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:38:10.070026 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac2ea3-ad2c-497a-a3c7-3444e051030d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:10.149267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8354a5-e7d0-4f30-bd0c-b8f76844d5fb None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:10.164081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:10.169747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:10.188754 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a5f0f-a3b7-42cc-9fe2-f2e37c07ba56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:10.189388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a5f0f-a3b7-42cc-9fe2-f2e37c07ba56 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:10.189663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a5f0f-a3b7-42cc-9fe2-f2e37c07ba56 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:10.189930 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a5f0f-a3b7-42cc-9fe2-f2e37c07ba56 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:10.190607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a5f0f-a3b7-42cc-9fe2-f2e37c07ba56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:10.190906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a5f0f-a3b7-42cc-9fe2-f2e37c07ba56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:10.191864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac2ea3-ad2c-497a-a3c7-3444e051030d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:10.192366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a5f0f-a3b7-42cc-9fe2-f2e37c07ba56 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:10.202397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 155/306] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:10.203655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef49a9-5ed6-4335-a7b0-a806dbf3586f None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:10.204034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef49a9-5ed6-4335-a7b0-a806dbf3586f None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:10.204309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef49a9-5ed6-4335-a7b0-a806dbf3586f None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:10.205633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 156/307] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:10.208391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef49a9-5ed6-4335-a7b0-a806dbf3586f None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:10.236588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8f6aa-428b-47c3-a980-0cf6e0b3719d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:10.237270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8f6aa-428b-47c3-a980-0cf6e0b3719d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:10.237736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8f6aa-428b-47c3-a980-0cf6e0b3719d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:10.237842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8f6aa-428b-47c3-a980-0cf6e0b3719d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:10.238918 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8f6aa-428b-47c3-a980-0cf6e0b3719d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:10.240432 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8f6aa-428b-47c3-a980-0cf6e0b3719d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:10.242345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 152/308] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:10.243523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2fc8a-a7bb-4abf-b1e5-03e045c0464a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:10.248918 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2fc8a-a7bb-4abf-b1e5-03e045c0464a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:10.249228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2fc8a-a7bb-4abf-b1e5-03e045c0464a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:10.267750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-31b2fc8a-a7bb-4abf-b1e5-03e045c0464a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:10.268935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-31b2fc8a-a7bb-4abf-b1e5-03e045c0464a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:10.320113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef49a9-5ed6-4335-a7b0-a806dbf3586f None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:10.341310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 153/309] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:10.353169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2fc8a-a7bb-4abf-b1e5-03e045c0464a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:10.354411 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1728b-3335-4d06-badb-642e933f8a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:10.355043 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1728b-3335-4d06-badb-642e933f8a52 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:10.355315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1728b-3335-4d06-badb-642e933f8a52 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:10.355579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1728b-3335-4d06-badb-642e933f8a52 None admin] PATH_INFO: `/v3/users/7e5665d05bb7449eba95c35f30126a9d` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:10.371342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1728b-3335-4d06-badb-642e933f8a52 None admin] RBAC: Authorizing `identity:get_user(user_id=7e5665d05bb7449eba95c35f30126a9d)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:10.372819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1728b-3335-4d06-badb-642e933f8a52 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:10.374763 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 157/310] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:10 2022] GET /identity/v3/users/7e5665d05bb7449eba95c35f30126a9d => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:10.381268 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1728b-3335-4d06-badb-642e933f8a52 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:10.423651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:10.452175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:10.478091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2fc8a-a7bb-4abf-b1e5-03e045c0464a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:10.508879 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1728b-3335-4d06-badb-642e933f8a52 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:10.525725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07f4b5-4aba-4f81-b010-923c82859117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:10.526517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07f4b5-4aba-4f81-b010-923c82859117 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:10.526882 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07f4b5-4aba-4f81-b010-923c82859117 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:10.527246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07f4b5-4aba-4f81-b010-923c82859117 None admin] PATH_INFO: `/v3/groups` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:10.528007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07f4b5-4aba-4f81-b010-923c82859117 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:10.529767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07f4b5-4aba-4f81-b010-923c82859117 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:10.548235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d39918-ecfb-494c-b502-07dcb1ab06fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:10.548849 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d39918-ecfb-494c-b502-07dcb1ab06fd None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:10.549111 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 154/311] 10.222.0.36 () {60 vars in 1134 bytes} [Thu Aug 18 14:38:10 2022] POST /identity/v3/groups => generated 229 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:38:10.549446 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d39918-ecfb-494c-b502-07dcb1ab06fd None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:10.549726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d39918-ecfb-494c-b502-07dcb1ab06fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:10.550590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d39918-ecfb-494c-b502-07dcb1ab06fd None admin] RBAC: Adding query filter params (role.id=a853ef2aaba2459f86b50cae95d23577, user.id=7e5665d05bb7449eba95c35f30126a9d, scope.system=all) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:10.550901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d39918-ecfb-494c-b502-07dcb1ab06fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:10.559730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d39918-ecfb-494c-b502-07dcb1ab06fd None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:10.565959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 158/312] 10.222.0.36 () {56 vars in 1308 bytes} [Thu Aug 18 14:38:10 2022] GET /identity/v3/role_assignments?user.id=7e5665d05bb7449eba95c35f30126a9d&scope.system=all&role.id=a853ef2aaba2459f86b50cae95d23577 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:10.628424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07f4b5-4aba-4f81-b010-923c82859117 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:10.745244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07f4b5-4aba-4f81-b010-923c82859117 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:10.782123 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650402c8-d718-42bf-88fe-996caf97f566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:10.782758 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650402c8-d718-42bf-88fe-996caf97f566 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:10.783034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650402c8-d718-42bf-88fe-996caf97f566 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:10.783303 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650402c8-d718-42bf-88fe-996caf97f566 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:10.794040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650402c8-d718-42bf-88fe-996caf97f566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-650402c8-d718-42bf-88fe-996caf97f566 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:10.799419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:10.800326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:10.801102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 155/313] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:10.807638 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d39918-ecfb-494c-b502-07dcb1ab06fd None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:10.908211 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:10.914297 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d39918-ecfb-494c-b502-07dcb1ab06fd None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:10.950630 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6a13c-5655-4d35-9430-61a734f5ef43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:10.951247 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6a13c-5655-4d35-9430-61a734f5ef43 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:10.951523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6a13c-5655-4d35-9430-61a734f5ef43 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:10.951824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6a13c-5655-4d35-9430-61a734f5ef43 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:10.952493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6a13c-5655-4d35-9430-61a734f5ef43 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:10.952794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6a13c-5655-4d35-9430-61a734f5ef43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:10.954715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6a13c-5655-4d35-9430-61a734f5ef43 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:10.963636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 159/314] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:10.969277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650402c8-d718-42bf-88fe-996caf97f566 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:10.974180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:11.082998 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650402c8-d718-42bf-88fe-996caf97f566 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:11.120061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44f912-9d16-456c-b6ab-0919d8062809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:11.120697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44f912-9d16-456c-b6ab-0919d8062809 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:11.120973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44f912-9d16-456c-b6ab-0919d8062809 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:11.121243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44f912-9d16-456c-b6ab-0919d8062809 None admin] PATH_INFO: `/v3/users/9f2be03176e24bd49439d5c12f122d47` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:11.122392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44f912-9d16-456c-b6ab-0919d8062809 None admin] RBAC: Authorizing `identity:get_user(user_id=9f2be03176e24bd49439d5c12f122d47)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:11.124159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44f912-9d16-456c-b6ab-0919d8062809 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:11.126537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 156/315] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:10 2022] GET /identity/v3/users/9f2be03176e24bd49439d5c12f122d47 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:11.131948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6a13c-5655-4d35-9430-61a734f5ef43 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:11.237862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6a13c-5655-4d35-9430-61a734f5ef43 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:11.272810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac16e59-cb12-44ad-8fd9-607e98f22a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:11.273429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac16e59-cb12-44ad-8fd9-607e98f22a68 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:11.273705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac16e59-cb12-44ad-8fd9-607e98f22a68 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:11.273977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac16e59-cb12-44ad-8fd9-607e98f22a68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:11.301459 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:11.343562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac16e59-cb12-44ad-8fd9-607e98f22a68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:11.345540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac16e59-cb12-44ad-8fd9-607e98f22a68 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:11.445609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 160/316] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:11.450936 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44f912-9d16-456c-b6ab-0919d8062809 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:11.523268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:11.556612 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44f912-9d16-456c-b6ab-0919d8062809 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:11.592297 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3073fa-82fb-40e8-9e42-581a32bbb0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:11.592926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3073fa-82fb-40e8-9e42-581a32bbb0f3 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:11.593207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3073fa-82fb-40e8-9e42-581a32bbb0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:11.593479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3073fa-82fb-40e8-9e42-581a32bbb0f3 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:11.594691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3073fa-82fb-40e8-9e42-581a32bbb0f3 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:11.596128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3073fa-82fb-40e8-9e42-581a32bbb0f3 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:11.598074 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 157/317] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:11 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:11.603960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac16e59-cb12-44ad-8fd9-607e98f22a68 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:11.663229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:11.712210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac16e59-cb12-44ad-8fd9-607e98f22a68 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:11.747378 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79607836-a721-49aa-9134-b11930373901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:11.748257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79607836-a721-49aa-9134-b11930373901 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:11.748537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79607836-a721-49aa-9134-b11930373901 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:11.748804 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79607836-a721-49aa-9134-b11930373901 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:11.749518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79607836-a721-49aa-9134-b11930373901 None admin] RBAC: Adding query filter params (role.id=a853ef2aaba2459f86b50cae95d23577, scope.project.id=c9ba9949b27349ab83f686eee37fe6f7, user.id=9f2be03176e24bd49439d5c12f122d47) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:11.749820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79607836-a721-49aa-9134-b11930373901 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:11.751829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79607836-a721-49aa-9134-b11930373901 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:11.758404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 161/318] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:11 2022] GET /identity/v3/role_assignments?user.id=9f2be03176e24bd49439d5c12f122d47&scope.project.id=c9ba9949b27349ab83f686eee37fe6f7&role.id=a853ef2aaba2459f86b50cae95d23577 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:12.038452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df535776-e50d-426a-b862-46f28d23f5ca None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:12.038772 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df535776-e50d-426a-b862-46f28d23f5ca None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:12.039016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df535776-e50d-426a-b862-46f28d23f5ca None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:12.040540 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 158/319] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:12.045476 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b632ae7-f298-449d-bf05-0407336cc0ff None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:12.045774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b632ae7-f298-449d-bf05-0407336cc0ff None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:12.046052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b632ae7-f298-449d-bf05-0407336cc0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:12.064527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-3b632ae7-f298-449d-bf05-0407336cc0ff None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:12.065712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3b632ae7-f298-449d-bf05-0407336cc0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:12.135404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 162/320] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:12.408277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df535776-e50d-426a-b862-46f28d23f5ca None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:12.512928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df535776-e50d-426a-b862-46f28d23f5ca None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:12.515802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a9641-dd5f-4447-81fe-a2e098a817ed None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:12.516089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a9641-dd5f-4447-81fe-a2e098a817ed None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:12.516403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a9641-dd5f-4447-81fe-a2e098a817ed None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:12.517749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 163/321] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:12.524704 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb0344-0d1e-494b-9d0e-bac831c8666f None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:12.525055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb0344-0d1e-494b-9d0e-bac831c8666f None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:12.525391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb0344-0d1e-494b-9d0e-bac831c8666f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:12.543781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-40eb0344-0d1e-494b-9d0e-bac831c8666f None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:12.545049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-40eb0344-0d1e-494b-9d0e-bac831c8666f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:12.550508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0de074-b2ec-4bdc-935b-2e7a50e056a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:12.551134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0de074-b2ec-4bdc-935b-2e7a50e056a3 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:12.551406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0de074-b2ec-4bdc-935b-2e7a50e056a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:12.551670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0de074-b2ec-4bdc-935b-2e7a50e056a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:12.558521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:12.564492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0de074-b2ec-4bdc-935b-2e7a50e056a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-1d0de074-b2ec-4bdc-935b-2e7a50e056a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:12.569831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:12.570718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:12.571523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 159/322] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:12.576684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0de074-b2ec-4bdc-935b-2e7a50e056a3 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:12.598988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:12.619726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 164/323] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:12.621097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c998494-9542-45fc-91bd-2da6ee1351d1 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:12.621376 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c998494-9542-45fc-91bd-2da6ee1351d1 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:12.621644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c998494-9542-45fc-91bd-2da6ee1351d1 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:12.622887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 165/324] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:12.628663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5fbcb-a312-425a-910d-a909c099b386 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:12.628942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5fbcb-a312-425a-910d-a909c099b386 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:12.629209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5fbcb-a312-425a-910d-a909c099b386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:12.645885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f1f5fbcb-a312-425a-910d-a909c099b386 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:12.647059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f1f5fbcb-a312-425a-910d-a909c099b386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:12.682370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0de074-b2ec-4bdc-935b-2e7a50e056a3 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:12.711975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 166/325] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:12.717145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580746a-d1d9-4795-810b-39d387ffede8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:12.717930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580746a-d1d9-4795-810b-39d387ffede8 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:12.718347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580746a-d1d9-4795-810b-39d387ffede8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:12.718756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580746a-d1d9-4795-810b-39d387ffede8 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:12.719577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580746a-d1d9-4795-810b-39d387ffede8 None admin] RBAC: Adding query filter params (name=member) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:12.720039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580746a-d1d9-4795-810b-39d387ffede8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:12.721567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580746a-d1d9-4795-810b-39d387ffede8 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:12.729674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 160/326] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:12.736033 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5fbcb-a312-425a-910d-a909c099b386 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:12.832287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5fbcb-a312-425a-910d-a909c099b386 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:12.845602 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:12.867569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e847662-069e-4276-a4b5-8073045f607c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:12.868425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e847662-069e-4276-a4b5-8073045f607c None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:12.868840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e847662-069e-4276-a4b5-8073045f607c None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:12.869248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e847662-069e-4276-a4b5-8073045f607c None admin] PATH_INFO: `/v3/users/297d5031504e4dfeb6f76be284f59de4` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:12.870485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e847662-069e-4276-a4b5-8073045f607c None admin] RBAC: Authorizing `identity:get_user(user_id=297d5031504e4dfeb6f76be284f59de4)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:12.872090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e847662-069e-4276-a4b5-8073045f607c None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:12.874042 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 167/327] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:12 2022] GET /identity/v3/users/297d5031504e4dfeb6f76be284f59de4 => generated 293 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:12.880115 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580746a-d1d9-4795-810b-39d387ffede8 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:12.888912 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e847662-069e-4276-a4b5-8073045f607c None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:12.947847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:12.996127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580746a-d1d9-4795-810b-39d387ffede8 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:12.999291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e847662-069e-4276-a4b5-8073045f607c None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:13.028170 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6f637-48be-460c-a1ac-7a3020ce6171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:13.028766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6f637-48be-460c-a1ac-7a3020ce6171 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:13.029041 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6f637-48be-460c-a1ac-7a3020ce6171 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:13.029312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6f637-48be-460c-a1ac-7a3020ce6171 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:13.030020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6f637-48be-460c-a1ac-7a3020ce6171 None admin] RBAC: Adding query filter params (role.id=603573d2738b49c389ad913208d2ced1, user.id=297d5031504e4dfeb6f76be284f59de4, scope.system=all) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:13.030323 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6f637-48be-460c-a1ac-7a3020ce6171 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:13.031602 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcbd09-ba37-44fd-9a17-1f895ab8c8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:13.031760 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6f637-48be-460c-a1ac-7a3020ce6171 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:13.032743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcbd09-ba37-44fd-9a17-1f895ab8c8ca None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:13.033024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcbd09-ba37-44fd-9a17-1f895ab8c8ca None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:13.033300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcbd09-ba37-44fd-9a17-1f895ab8c8ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:13.043439 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcbd09-ba37-44fd-9a17-1f895ab8c8ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:13.044034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 161/328] 10.222.0.36 () {56 vars in 1308 bytes} [Thu Aug 18 14:38:12 2022] GET /identity/v3/role_assignments?user.id=297d5031504e4dfeb6f76be284f59de4&scope.system=all&role.id=603573d2738b49c389ad913208d2ced1 => generated 510 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:13.044034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6f637-48be-460c-a1ac-7a3020ce6171 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-28dcbd09-ba37-44fd-9a17-1f895ab8c8ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:13.051848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:13.052843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:13.053639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 168/329] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:13.059226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcbd09-ba37-44fd-9a17-1f895ab8c8ca None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:13.115754 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:13.155878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6f637-48be-460c-a1ac-7a3020ce6171 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:13.177744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:13.186324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcbd09-ba37-44fd-9a17-1f895ab8c8ca None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:13.199490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc823c-5723-417e-8801-99e2e2a0e021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:13.200082 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc823c-5723-417e-8801-99e2e2a0e021 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:13.200359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc823c-5723-417e-8801-99e2e2a0e021 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:13.200624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc823c-5723-417e-8801-99e2e2a0e021 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:13.202260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc823c-5723-417e-8801-99e2e2a0e021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:13.205419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc823c-5723-417e-8801-99e2e2a0e021 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:13.205419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 162/330] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:13.211185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc823c-5723-417e-8801-99e2e2a0e021 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:13.230511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562818b-6eef-40f0-96a2-d19bd197c5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:13.230969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562818b-6eef-40f0-96a2-d19bd197c5d7 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:13.231260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562818b-6eef-40f0-96a2-d19bd197c5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:13.231531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562818b-6eef-40f0-96a2-d19bd197c5d7 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:13.232286 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562818b-6eef-40f0-96a2-d19bd197c5d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:13.232589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562818b-6eef-40f0-96a2-d19bd197c5d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:13.233976 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562818b-6eef-40f0-96a2-d19bd197c5d7 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:13.242468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 169/331] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:13.250260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562818b-6eef-40f0-96a2-d19bd197c5d7 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:13.317794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc823c-5723-417e-8801-99e2e2a0e021 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:13.348997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562818b-6eef-40f0-96a2-d19bd197c5d7 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:13.350573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:13.351454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5bc21-d987-4193-9332-f120aa733859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:13.352215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5bc21-d987-4193-9332-f120aa733859 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:13.352527 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5bc21-d987-4193-9332-f120aa733859 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:13.352958 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5bc21-d987-4193-9332-f120aa733859 None admin] PATH_INFO: `/v3/groups` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:13.353587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5bc21-d987-4193-9332-f120aa733859 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:13.355047 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5bc21-d987-4193-9332-f120aa733859 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:13.369439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 163/332] 10.222.0.36 () {60 vars in 1134 bytes} [Thu Aug 18 14:38:13 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:38:13.386468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a5885-1a4b-4adb-bb81-f21385a62ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:13.387123 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a5885-1a4b-4adb-bb81-f21385a62ef5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:13.387400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a5885-1a4b-4adb-bb81-f21385a62ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:13.387670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a5885-1a4b-4adb-bb81-f21385a62ef5 None admin] PATH_INFO: `/v3/users/7e5665d05bb7449eba95c35f30126a9d` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:13.388910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a5885-1a4b-4adb-bb81-f21385a62ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=7e5665d05bb7449eba95c35f30126a9d)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:13.390389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a5885-1a4b-4adb-bb81-f21385a62ef5 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:13.392616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 170/333] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:13 2022] GET /identity/v3/users/7e5665d05bb7449eba95c35f30126a9d => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:13.398637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5bc21-d987-4193-9332-f120aa733859 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:13.449287 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:13.508258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5bc21-d987-4193-9332-f120aa733859 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:13.543249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725abb7b-e0d6-4fad-b6f0-21dd8ba3fd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:13.543922 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725abb7b-e0d6-4fad-b6f0-21dd8ba3fd35 None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:13.544203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725abb7b-e0d6-4fad-b6f0-21dd8ba3fd35 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:13.544480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725abb7b-e0d6-4fad-b6f0-21dd8ba3fd35 None admin] PATH_INFO: `/v3/system/users/7e5665d05bb7449eba95c35f30126a9d/roles/a853ef2aaba2459f86b50cae95d23577` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:13.545897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725abb7b-e0d6-4fad-b6f0-21dd8ba3fd35 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7e5665d05bb7449eba95c35f30126a9d, role_id=a853ef2aaba2459f86b50cae95d23577)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:13.547288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725abb7b-e0d6-4fad-b6f0-21dd8ba3fd35 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:13.555067 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 164/334] 10.222.0.36 () {58 vars in 1334 bytes} [Thu Aug 18 14:38:13 2022] PUT /identity/v3/system/users/7e5665d05bb7449eba95c35f30126a9d/roles/a853ef2aaba2459f86b50cae95d23577 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:15.665193 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9bc85-a898-4ee6-9324-76316ca73410 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:15.666121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9bc85-a898-4ee6-9324-76316ca73410 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:15.666573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9bc85-a898-4ee6-9324-76316ca73410 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:15.668075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 171/335] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:15.678478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257755d0-bcc8-4029-a660-01a11ea56d45 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:15.678945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257755d0-bcc8-4029-a660-01a11ea56d45 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:15.679410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257755d0-bcc8-4029-a660-01a11ea56d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:15.696141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1a112-bff8-4433-8cc0-cfa9774c7c86 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:15.696718 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1a112-bff8-4433-8cc0-cfa9774c7c86 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:15.697208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1a112-bff8-4433-8cc0-cfa9774c7c86 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:15.698993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 172/336] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:15.703638 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc2b0a-baea-49a9-8eeb-ab770c0d6f5a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:15.704229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc2b0a-baea-49a9-8eeb-ab770c0d6f5a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:15.704701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc2b0a-baea-49a9-8eeb-ab770c0d6f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:15.720318 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-257755d0-bcc8-4029-a660-01a11ea56d45 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:15.722319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-257755d0-bcc8-4029-a660-01a11ea56d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:15.732670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-43fc2b0a-baea-49a9-8eeb-ab770c0d6f5a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:15.734154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-43fc2b0a-baea-49a9-8eeb-ab770c0d6f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:15.897329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 165/338] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:15.897518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 173/338] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:15.898684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f70423-be31-440d-9a6b-d095205a8a5b None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:15.899130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f70423-be31-440d-9a6b-d095205a8a5b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:15.899565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f70423-be31-440d-9a6b-d095205a8a5b None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:15.901184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 166/339] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:15.909982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308139-0ec2-4b67-abf8-9a36f809fd7e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:15.910470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308139-0ec2-4b67-abf8-9a36f809fd7e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:15.910953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308139-0ec2-4b67-abf8-9a36f809fd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:15.970852 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-9c308139-0ec2-4b67-abf8-9a36f809fd7e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:15.972843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9c308139-0ec2-4b67-abf8-9a36f809fd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:16.093328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 174/340] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:16.437736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f70423-be31-440d-9a6b-d095205a8a5b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:16.477041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c308139-0ec2-4b67-abf8-9a36f809fd7e None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:16.489861 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:16.533319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:16.568379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f70423-be31-440d-9a6b-d095205a8a5b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:16.621834 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58eaa70-e239-424d-bfe9-fd30cf65b1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:16.622797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58eaa70-e239-424d-bfe9-fd30cf65b1d3 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:16.623554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58eaa70-e239-424d-bfe9-fd30cf65b1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:16.624062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58eaa70-e239-424d-bfe9-fd30cf65b1d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:16.636848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58eaa70-e239-424d-bfe9-fd30cf65b1d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-f58eaa70-e239-424d-bfe9-fd30cf65b1d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:16.641981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:16.642851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:16.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:16.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:16.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:16.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:16.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:16.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:16.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:16.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:16.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:16.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:16.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 14:38:16.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:16.646201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 167/341] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:16.647199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58eaa70-e239-424d-bfe9-fd30cf65b1d3 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:16.669393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c308139-0ec2-4b67-abf8-9a36f809fd7e None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:16.727273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88531c59-d642-49de-812a-efc9b3f91bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:16.727888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88531c59-d642-49de-812a-efc9b3f91bd9 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:16.728170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88531c59-d642-49de-812a-efc9b3f91bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:16.728441 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88531c59-d642-49de-812a-efc9b3f91bd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:16.744840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88531c59-d642-49de-812a-efc9b3f91bd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-88531c59-d642-49de-812a-efc9b3f91bd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:16.749942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:16.751366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:16.752205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:16.752205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:16.752205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:16.752205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:16.752205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:16.752205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:16.752205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:16.752205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:16.752205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:16.752205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:16.752205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:16.752205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:16.754372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58eaa70-e239-424d-bfe9-fd30cf65b1d3 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:16.756005 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:16.759074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 175/342] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:38:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:16.760654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88531c59-d642-49de-812a-efc9b3f91bd9 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:16.795045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94930127-e7e2-4ec1-b6d8-bd1c958dba0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:16.795673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94930127-e7e2-4ec1-b6d8-bd1c958dba0d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:16.795972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94930127-e7e2-4ec1-b6d8-bd1c958dba0d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:16.796241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94930127-e7e2-4ec1-b6d8-bd1c958dba0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:16.819992 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94930127-e7e2-4ec1-b6d8-bd1c958dba0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-94930127-e7e2-4ec1-b6d8-bd1c958dba0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:16.825424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:16.826345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:16.827171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 168/343] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:16.830721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94930127-e7e2-4ec1-b6d8-bd1c958dba0d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:16.939257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:16.949722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88531c59-d642-49de-812a-efc9b3f91bd9 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:16.992911 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1d7e0-1f1c-4205-adac-d3b978caf82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:16.993443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1d7e0-1f1c-4205-adac-d3b978caf82d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:16.993720 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1d7e0-1f1c-4205-adac-d3b978caf82d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:16.993989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1d7e0-1f1c-4205-adac-d3b978caf82d None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:16.995039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1d7e0-1f1c-4205-adac-d3b978caf82d None admin] RBAC: Adding query filter params (name=reader) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:16.995039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1d7e0-1f1c-4205-adac-d3b978caf82d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:16.996462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1d7e0-1f1c-4205-adac-d3b978caf82d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:17.007345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 176/344] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:17.010620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1d7e0-1f1c-4205-adac-d3b978caf82d None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:17.013837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94930127-e7e2-4ec1-b6d8-bd1c958dba0d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:17.057284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48ff74-ea1b-4de5-95c3-71b99dde850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:17.057781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48ff74-ea1b-4de5-95c3-71b99dde850b None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:17.058155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48ff74-ea1b-4de5-95c3-71b99dde850b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:17.058525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48ff74-ea1b-4de5-95c3-71b99dde850b None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:17.059288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48ff74-ea1b-4de5-95c3-71b99dde850b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:17.059706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48ff74-ea1b-4de5-95c3-71b99dde850b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:17.061298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48ff74-ea1b-4de5-95c3-71b99dde850b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:17.073873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 169/345] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:17.075029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb940d-dea7-43f0-b3d3-d804f1836cda None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:17.075407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb940d-dea7-43f0-b3d3-d804f1836cda None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:17.075809 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb940d-dea7-43f0-b3d3-d804f1836cda None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:17.077136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 170/346] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:17.077923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb940d-dea7-43f0-b3d3-d804f1836cda None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:17.110513 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:17.144897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1d7e0-1f1c-4205-adac-d3b978caf82d None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:17.191044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f1e46-4067-4d11-b6b0-b724cbd59396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:17.191931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f1e46-4067-4d11-b6b0-b724cbd59396 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:17.192654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f1e46-4067-4d11-b6b0-b724cbd59396 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:17.192938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f1e46-4067-4d11-b6b0-b724cbd59396 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:17.193899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f1e46-4067-4d11-b6b0-b724cbd59396 None admin] RBAC: Adding query filter params (name=member) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:17.194208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f1e46-4067-4d11-b6b0-b724cbd59396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:17.196267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f1e46-4067-4d11-b6b0-b724cbd59396 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:17.207182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 177/347] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:17.208202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f1e46-4067-4d11-b6b0-b724cbd59396 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:17.208941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb940d-dea7-43f0-b3d3-d804f1836cda None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:17.230499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:17.253712 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4efd2-81dc-4b2e-8c90-e595623bbbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:17.254385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4efd2-81dc-4b2e-8c90-e595623bbbd8 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:17.254669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4efd2-81dc-4b2e-8c90-e595623bbbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:17.255053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4efd2-81dc-4b2e-8c90-e595623bbbd8 None admin] PATH_INFO: `/v3/users/7e5665d05bb7449eba95c35f30126a9d` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:17.256304 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4efd2-81dc-4b2e-8c90-e595623bbbd8 None admin] RBAC: Authorizing `identity:get_user(user_id=7e5665d05bb7449eba95c35f30126a9d)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:17.257790 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4efd2-81dc-4b2e-8c90-e595623bbbd8 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:17.259756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 171/348] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:38:17 2022] GET /identity/v3/users/7e5665d05bb7449eba95c35f30126a9d => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:17.261041 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43944349-cea0-4e18-b094-1088097039c7 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:17.261388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43944349-cea0-4e18-b094-1088097039c7 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:17.261651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43944349-cea0-4e18-b094-1088097039c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:17.283252 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-43944349-cea0-4e18-b094-1088097039c7 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:17.284525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-43944349-cea0-4e18-b094-1088097039c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:17.348525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f1e46-4067-4d11-b6b0-b724cbd59396 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:17.371916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 172/349] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:17.372796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43944349-cea0-4e18-b094-1088097039c7 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:17.408430 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf992e-f714-411f-a70f-4ec9a801a4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:17.409115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf992e-f714-411f-a70f-4ec9a801a4d0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:17.409399 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf992e-f714-411f-a70f-4ec9a801a4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:17.409673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf992e-f714-411f-a70f-4ec9a801a4d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:17.414699 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:17.538584 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf992e-f714-411f-a70f-4ec9a801a4d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:17.540029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf992e-f714-411f-a70f-4ec9a801a4d0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:17.557061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43944349-cea0-4e18-b094-1088097039c7 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:17.557450 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:17.610833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746248c7-9e87-46a9-94b5-684c4e978e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:17.612172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746248c7-9e87-46a9-94b5-684c4e978e02 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:17.612651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746248c7-9e87-46a9-94b5-684c4e978e02 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:17.613114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746248c7-9e87-46a9-94b5-684c4e978e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:17.668892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 178/350] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:17.669845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf992e-f714-411f-a70f-4ec9a801a4d0 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:17.706155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746248c7-9e87-46a9-94b5-684c4e978e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:17.707764 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746248c7-9e87-46a9-94b5-684c4e978e02 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:17.735857 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:17.806636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf992e-f714-411f-a70f-4ec9a801a4d0 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:17.824891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:17.838017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 173/351] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:17.839371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f2da9-2c64-4079-8637-9b3c56889ded None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:17.839658 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f2da9-2c64-4079-8637-9b3c56889ded None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:17.839953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f2da9-2c64-4079-8637-9b3c56889ded None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:17.841342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 174/352] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:17.841890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f2da9-2c64-4079-8637-9b3c56889ded None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:17.855402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2833-344f-4884-8d22-89a41d65a65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:17.856110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2833-344f-4884-8d22-89a41d65a65d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:17.856388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2833-344f-4884-8d22-89a41d65a65d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:17.856722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2833-344f-4884-8d22-89a41d65a65d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:17.857453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2833-344f-4884-8d22-89a41d65a65d None admin] RBAC: Adding query filter params (role.id=a853ef2aaba2459f86b50cae95d23577, user.id=7e5665d05bb7449eba95c35f30126a9d, scope.system=all) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:17.857757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2833-344f-4884-8d22-89a41d65a65d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:17.859185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2833-344f-4884-8d22-89a41d65a65d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:17.872690 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 179/353] 10.222.0.36 () {56 vars in 1308 bytes} [Thu Aug 18 14:38:17 2022] GET /identity/v3/role_assignments?user.id=7e5665d05bb7449eba95c35f30126a9d&scope.system=all&role.id=a853ef2aaba2459f86b50cae95d23577 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:17.874308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2833-344f-4884-8d22-89a41d65a65d None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:17.993322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f2da9-2c64-4079-8637-9b3c56889ded None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:18.011843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2833-344f-4884-8d22-89a41d65a65d None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:18.023221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:18.042168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c19073-101c-4a52-9fc2-6d1b40e76698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:18.043111 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c19073-101c-4a52-9fc2-6d1b40e76698 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.043545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c19073-101c-4a52-9fc2-6d1b40e76698 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.043979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c19073-101c-4a52-9fc2-6d1b40e76698 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.045305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c19073-101c-4a52-9fc2-6d1b40e76698 None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:18.047105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c19073-101c-4a52-9fc2-6d1b40e76698 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:18.050141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 175/354] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:17 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:18.051478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d233b-d1cf-4112-9a1b-ce2c4f891b84 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.051931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d233b-d1cf-4112-9a1b-ce2c4f891b84 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.052344 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d233b-d1cf-4112-9a1b-ce2c4f891b84 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.053779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 176/355] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:18.054931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707e2e8-5a77-443e-be9e-2732282ae3d0 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.055366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707e2e8-5a77-443e-be9e-2732282ae3d0 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.055847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707e2e8-5a77-443e-be9e-2732282ae3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.059613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3eed17-09e9-4c0d-84e8-a5310d4f09c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:18.060233 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eed17-09e9-4c0d-84e8-a5310d4f09c5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.060515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eed17-09e9-4c0d-84e8-a5310d4f09c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.060787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eed17-09e9-4c0d-84e8-a5310d4f09c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.075630 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3eed17-09e9-4c0d-84e8-a5310d4f09c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:18.077371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8707e2e8-5a77-443e-be9e-2732282ae3d0 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:18.078824 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8707e2e8-5a77-443e-be9e-2732282ae3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-3f3eed17-09e9-4c0d-84e8-a5310d4f09c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:18.081585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:18.082487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:18.083291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 180/356] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:18.083984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3eed17-09e9-4c0d-84e8-a5310d4f09c5 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:18.124151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:18.165189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 177/357] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:18.165876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707e2e8-5a77-443e-be9e-2732282ae3d0 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:18.214047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3eed17-09e9-4c0d-84e8-a5310d4f09c5 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:18.260205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211b93e-7e55-4998-9161-2bf8389d224e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:18.261312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211b93e-7e55-4998-9161-2bf8389d224e None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.261597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211b93e-7e55-4998-9161-2bf8389d224e None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.261871 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211b93e-7e55-4998-9161-2bf8389d224e None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.263343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211b93e-7e55-4998-9161-2bf8389d224e None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:18.264861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211b93e-7e55-4998-9161-2bf8389d224e None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:18.267121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 181/358] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:18 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:18.268278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4339a-bf3c-4c2c-9e5c-f24f5ffe48ba None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.268560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4339a-bf3c-4c2c-9e5c-f24f5ffe48ba None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.268831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4339a-bf3c-4c2c-9e5c-f24f5ffe48ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.287100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-78e4339a-bf3c-4c2c-9e5c-f24f5ffe48ba None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:18.288047 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707e2e8-5a77-443e-be9e-2732282ae3d0 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:18.288332 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-78e4339a-bf3c-4c2c-9e5c-f24f5ffe48ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:18.310451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:18.330548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c318a-a807-44fb-85a6-c722c57d2b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:18.331188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c318a-a807-44fb-85a6-c722c57d2b1c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.331473 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c318a-a807-44fb-85a6-c722c57d2b1c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.331807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c318a-a807-44fb-85a6-c722c57d2b1c None admin] PATH_INFO: `/v3/groups/00df45b9a8764d11a3d06bc6c30efe31` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.340957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c318a-a807-44fb-85a6-c722c57d2b1c None admin] RBAC: Authorizing `identity:get_group(group_id=00df45b9a8764d11a3d06bc6c30efe31)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:18.343013 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c318a-a807-44fb-85a6-c722c57d2b1c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:18.345359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 178/359] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:18 2022] GET /identity/v3/groups/00df45b9a8764d11a3d06bc6c30efe31 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:18.346309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c318a-a807-44fb-85a6-c722c57d2b1c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:18.369533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 182/360] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:18.370387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4339a-bf3c-4c2c-9e5c-f24f5ffe48ba None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:18.408583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:18.460981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c318a-a807-44fb-85a6-c722c57d2b1c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:18.486573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4339a-bf3c-4c2c-9e5c-f24f5ffe48ba None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:18.503025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61fd1a-6c53-420e-992e-ee9455a5ba2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:18.503636 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61fd1a-6c53-420e-992e-ee9455a5ba2d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.505963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61fd1a-6c53-420e-992e-ee9455a5ba2d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.506253 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61fd1a-6c53-420e-992e-ee9455a5ba2d None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.507575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61fd1a-6c53-420e-992e-ee9455a5ba2d None admin] RBAC: Adding query filter params (name=member) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:18.507575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61fd1a-6c53-420e-992e-ee9455a5ba2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:18.508659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61fd1a-6c53-420e-992e-ee9455a5ba2d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:18.517763 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 179/361] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:18.518647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61fd1a-6c53-420e-992e-ee9455a5ba2d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:18.531572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c832c3-d4e3-44fc-8581-e0f0657116c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:18.532779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c832c3-d4e3-44fc-8581-e0f0657116c1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.533265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c832c3-d4e3-44fc-8581-e0f0657116c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.533877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c832c3-d4e3-44fc-8581-e0f0657116c1 None admin] PATH_INFO: `/v3/groups/00df45b9a8764d11a3d06bc6c30efe31` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.535485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c832c3-d4e3-44fc-8581-e0f0657116c1 None admin] RBAC: Authorizing `identity:get_group(group_id=00df45b9a8764d11a3d06bc6c30efe31)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:18.537930 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c832c3-d4e3-44fc-8581-e0f0657116c1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:18.540691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 183/362] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:18 2022] GET /identity/v3/groups/00df45b9a8764d11a3d06bc6c30efe31 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:18.541756 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c832c3-d4e3-44fc-8581-e0f0657116c1 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:18.571975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:18.652235 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61fd1a-6c53-420e-992e-ee9455a5ba2d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:18.668749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:18.673106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c832c3-d4e3-44fc-8581-e0f0657116c1 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:18.692788 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d4511-0f98-4eb3-aac6-bb44459a0f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:18.693401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d4511-0f98-4eb3-aac6-bb44459a0f52 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.693689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d4511-0f98-4eb3-aac6-bb44459a0f52 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.693961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d4511-0f98-4eb3-aac6-bb44459a0f52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.694693 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d4511-0f98-4eb3-aac6-bb44459a0f52 None admin] RBAC: Adding query filter params (group.id=00df45b9a8764d11a3d06bc6c30efe31, role.id=9df5a7d7691c49b9a78f0db0a050b7fe, scope.project.id=c37e716487e9421694a14be7fa5c2d0c) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:18.695000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d4511-0f98-4eb3-aac6-bb44459a0f52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:18.696470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d4511-0f98-4eb3-aac6-bb44459a0f52 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:18.702278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 180/363] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Aug 18 14:38:18 2022] GET /identity/v3/role_assignments?group.id=00df45b9a8764d11a3d06bc6c30efe31&scope.project.id=c37e716487e9421694a14be7fa5c2d0c&role.id=9df5a7d7691c49b9a78f0db0a050b7fe => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:18.703079 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d4511-0f98-4eb3-aac6-bb44459a0f52 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:18.709899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8fdce-08f6-45e0-9775-69f8e8a0c787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:18.710556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8fdce-08f6-45e0-9775-69f8e8a0c787 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.710836 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8fdce-08f6-45e0-9775-69f8e8a0c787 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.711107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8fdce-08f6-45e0-9775-69f8e8a0c787 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.723327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8fdce-08f6-45e0-9775-69f8e8a0c787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-bbc8fdce-08f6-45e0-9775-69f8e8a0c787 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:18.729617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:18.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 184/364] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:38:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:18.731507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8fdce-08f6-45e0-9775-69f8e8a0c787 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:18.851030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d4511-0f98-4eb3-aac6-bb44459a0f52 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:18.862038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:18.879203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8fdce-08f6-45e0-9775-69f8e8a0c787 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:18.897703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb6f2a-0e04-4078-bd71-90f1096b87a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:18.898326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb6f2a-0e04-4078-bd71-90f1096b87a1 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.898601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb6f2a-0e04-4078-bd71-90f1096b87a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.898868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb6f2a-0e04-4078-bd71-90f1096b87a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.926658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1cd01-fa96-44d5-a40f-071841625725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:18.927271 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1cd01-fa96-44d5-a40f-071841625725 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:18.927545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1cd01-fa96-44d5-a40f-071841625725 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:18.927842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1cd01-fa96-44d5-a40f-071841625725 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:18.928581 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1cd01-fa96-44d5-a40f-071841625725 None admin] RBAC: Adding query filter params (group.id=00df45b9a8764d11a3d06bc6c30efe31, role.id=603573d2738b49c389ad913208d2ced1, scope.project.id=c37e716487e9421694a14be7fa5c2d0c) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:18.928886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1cd01-fa96-44d5-a40f-071841625725 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:18.930318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1cd01-fa96-44d5-a40f-071841625725 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:18.936746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 185/365] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Aug 18 14:38:18 2022] GET /identity/v3/role_assignments?group.id=00df45b9a8764d11a3d06bc6c30efe31&scope.project.id=c37e716487e9421694a14be7fa5c2d0c&role.id=603573d2738b49c389ad913208d2ced1 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:18.939168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1cd01-fa96-44d5-a40f-071841625725 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:18.971458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:19.003422 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb6f2a-0e04-4078-bd71-90f1096b87a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:19.005300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb6f2a-0e04-4078-bd71-90f1096b87a1 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:19.062719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1cd01-fa96-44d5-a40f-071841625725 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:19.099518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006b9ba-8399-4880-a753-3c948b687789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:19.100425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006b9ba-8399-4880-a753-3c948b687789 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:19.100874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006b9ba-8399-4880-a753-3c948b687789 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:19.101335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006b9ba-8399-4880-a753-3c948b687789 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:19.113465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006b9ba-8399-4880-a753-3c948b687789 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:19.118322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 181/366] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:19.119199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb6f2a-0e04-4078-bd71-90f1096b87a1 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-2006b9ba-8399-4880-a753-3c948b687789 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:19.119412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:19.120359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:19.121260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:19.121260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:19.121260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:19.121260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:19.121260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:19.121260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:19.121260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:19.121260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:19.121260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:19.121260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:19.121260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:38:19.121260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:19.127778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 186/367] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:38:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:19.128879 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006b9ba-8399-4880-a753-3c948b687789 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:19.130476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:19.214586 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:19.227425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb6f2a-0e04-4078-bd71-90f1096b87a1 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:19.235552 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006b9ba-8399-4880-a753-3c948b687789 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:19.265674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dcc1-5cc3-4355-b5a4-1088b965223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:19.266613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dcc1-5cc3-4355-b5a4-1088b965223e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:19.266925 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dcc1-5cc3-4355-b5a4-1088b965223e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:19.267194 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dcc1-5cc3-4355-b5a4-1088b965223e None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:19.267876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93dcc1-5cc3-4355-b5a4-1088b965223e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:19.268174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93dcc1-5cc3-4355-b5a4-1088b965223e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:19.269774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93dcc1-5cc3-4355-b5a4-1088b965223e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:19.272110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548643cf-c35f-4b7c-9d5b-270153d05749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:19.272918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548643cf-c35f-4b7c-9d5b-270153d05749 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:19.273334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548643cf-c35f-4b7c-9d5b-270153d05749 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:19.273741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548643cf-c35f-4b7c-9d5b-270153d05749 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:19.275019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548643cf-c35f-4b7c-9d5b-270153d05749 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:19.276609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548643cf-c35f-4b7c-9d5b-270153d05749 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:19.278780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 182/369] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:19.278952 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 187/369] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:19 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:19.279613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dcc1-5cc3-4355-b5a4-1088b965223e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:19.287080 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548643cf-c35f-4b7c-9d5b-270153d05749 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:19.412919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548643cf-c35f-4b7c-9d5b-270153d05749 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:19.424684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:19.456201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbbdc3-a837-4f35-9453-cea93fe68cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:19.456843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbbdc3-a837-4f35-9453-cea93fe68cfd None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:19.457125 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbbdc3-a837-4f35-9453-cea93fe68cfd None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:19.457396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbbdc3-a837-4f35-9453-cea93fe68cfd None admin] PATH_INFO: `/v3/groups/00df45b9a8764d11a3d06bc6c30efe31` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:19.459227 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbbdc3-a837-4f35-9453-cea93fe68cfd None admin] RBAC: Authorizing `identity:get_group(group_id=00df45b9a8764d11a3d06bc6c30efe31)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:19.460808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbbdc3-a837-4f35-9453-cea93fe68cfd None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:19.462934 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 188/370] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:19 2022] GET /identity/v3/groups/00df45b9a8764d11a3d06bc6c30efe31 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:19.463753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbbdc3-a837-4f35-9453-cea93fe68cfd None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:19.464384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dcc1-5cc3-4355-b5a4-1088b965223e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:19.489249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:19.512292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5018ac-2599-4722-be19-5af5d4b8b55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:19.512912 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5018ac-2599-4722-be19-5af5d4b8b55d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:19.513189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5018ac-2599-4722-be19-5af5d4b8b55d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:19.513461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5018ac-2599-4722-be19-5af5d4b8b55d None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:19.514131 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5018ac-2599-4722-be19-5af5d4b8b55d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:19.514434 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5018ac-2599-4722-be19-5af5d4b8b55d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:19.516919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5018ac-2599-4722-be19-5af5d4b8b55d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:19.528104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 183/371] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:19.528952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5018ac-2599-4722-be19-5af5d4b8b55d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:19.589093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbbdc3-a837-4f35-9453-cea93fe68cfd None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:19.628867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbc4c4-6420-4a8f-a8e6-4434e8c02dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:19.629581 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbc4c4-6420-4a8f-a8e6-4434e8c02dc5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:19.629864 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbc4c4-6420-4a8f-a8e6-4434e8c02dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:19.630269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbc4c4-6420-4a8f-a8e6-4434e8c02dc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:19.652321 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5018ac-2599-4722-be19-5af5d4b8b55d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:19.689079 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:19.696054 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f43ac0-8953-455f-8bcc-325875aa8840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:19.697018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f43ac0-8953-455f-8bcc-325875aa8840 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:19.697421 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f43ac0-8953-455f-8bcc-325875aa8840 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:19.697705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f43ac0-8953-455f-8bcc-325875aa8840 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:19.698573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f43ac0-8953-455f-8bcc-325875aa8840 None admin] RBAC: Adding query filter params (group.id=00df45b9a8764d11a3d06bc6c30efe31, role.id=603573d2738b49c389ad913208d2ced1, scope.project.id=c9ba9949b27349ab83f686eee37fe6f7) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:19.698898 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f43ac0-8953-455f-8bcc-325875aa8840 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:19.700990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f43ac0-8953-455f-8bcc-325875aa8840 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:19.711790 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 184/372] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Aug 18 14:38:19 2022] GET /identity/v3/role_assignments?group.id=00df45b9a8764d11a3d06bc6c30efe31&scope.project.id=c9ba9949b27349ab83f686eee37fe6f7&role.id=603573d2738b49c389ad913208d2ced1 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:19.712816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f43ac0-8953-455f-8bcc-325875aa8840 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:19.717485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbc4c4-6420-4a8f-a8e6-4434e8c02dc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:19.719067 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbc4c4-6420-4a8f-a8e6-4434e8c02dc5 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:19.760696 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:19.915704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f43ac0-8953-455f-8bcc-325875aa8840 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:19.928569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 189/373] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:19.935312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbc4c4-6420-4a8f-a8e6-4434e8c02dc5 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:19.981860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b678e9b-f301-4345-b591-ddf235513e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:19.982803 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b678e9b-f301-4345-b591-ddf235513e5f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:19.983553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b678e9b-f301-4345-b591-ddf235513e5f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:19.984066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b678e9b-f301-4345-b591-ddf235513e5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:20.067886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:20.076627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b678e9b-f301-4345-b591-ddf235513e5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:20.079981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b678e9b-f301-4345-b591-ddf235513e5f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:20.097699 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbc4c4-6420-4a8f-a8e6-4434e8c02dc5 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:20.129321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:20.154875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada01f5f-b7cc-4944-ac65-58df81b9df98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:20.155798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada01f5f-b7cc-4944-ac65-58df81b9df98 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:20.156216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada01f5f-b7cc-4944-ac65-58df81b9df98 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:20.156625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada01f5f-b7cc-4944-ac65-58df81b9df98 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:20.158554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada01f5f-b7cc-4944-ac65-58df81b9df98 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:20.160336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada01f5f-b7cc-4944-ac65-58df81b9df98 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:20.162110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 190/374] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:19 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:20.168794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada01f5f-b7cc-4944-ac65-58df81b9df98 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:20.200633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 185/375] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:20.209126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b678e9b-f301-4345-b591-ddf235513e5f None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:20.322041 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:20.326634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b678e9b-f301-4345-b591-ddf235513e5f None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:20.367222 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeca398-c9b3-4b82-9cc5-4177582046e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:20.369496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeca398-c9b3-4b82-9cc5-4177582046e9 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:20.369496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeca398-c9b3-4b82-9cc5-4177582046e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:20.369496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeca398-c9b3-4b82-9cc5-4177582046e9 None admin] PATH_INFO: `/v3/projects/fa81e49fae72473b99b445418d4fe5de` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:20.370548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeca398-c9b3-4b82-9cc5-4177582046e9 None admin] RBAC: Authorizing `identity:get_project(project_id=fa81e49fae72473b99b445418d4fe5de)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:20.373393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeca398-c9b3-4b82-9cc5-4177582046e9 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:20.379669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 186/376] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:20 2022] GET /identity/v3/projects/fa81e49fae72473b99b445418d4fe5de => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:20.381382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeca398-c9b3-4b82-9cc5-4177582046e9 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:20.412147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada01f5f-b7cc-4944-ac65-58df81b9df98 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:20.452391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78d03e-1d79-4448-b0ef-b1086b951adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:20.453992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78d03e-1d79-4448-b0ef-b1086b951adb None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:20.454691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78d03e-1d79-4448-b0ef-b1086b951adb None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:20.455373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78d03e-1d79-4448-b0ef-b1086b951adb None admin] PATH_INFO: `/v3/groups/00df45b9a8764d11a3d06bc6c30efe31` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:20.456854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78d03e-1d79-4448-b0ef-b1086b951adb None admin] RBAC: Authorizing `identity:get_group(group_id=00df45b9a8764d11a3d06bc6c30efe31)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:20.458578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78d03e-1d79-4448-b0ef-b1086b951adb None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:20.460445 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 191/377] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:20 2022] GET /identity/v3/groups/00df45b9a8764d11a3d06bc6c30efe31 => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:20.467467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78d03e-1d79-4448-b0ef-b1086b951adb None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:20.537419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:20.573164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeca398-c9b3-4b82-9cc5-4177582046e9 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:20.617766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78d03e-1d79-4448-b0ef-b1086b951adb None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:20.647568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa55208-32e7-4b3c-b981-84f43b6e90fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:20.648616 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa55208-32e7-4b3c-b981-84f43b6e90fb None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:20.650355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa55208-32e7-4b3c-b981-84f43b6e90fb None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:20.650880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa55208-32e7-4b3c-b981-84f43b6e90fb None admin] PATH_INFO: `/v3/groups/913b85621ab741538ee206002c163a1e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:20.662861 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa55208-32e7-4b3c-b981-84f43b6e90fb None admin] RBAC: Authorizing `identity:get_group(group_id=913b85621ab741538ee206002c163a1e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:20.664650 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa55208-32e7-4b3c-b981-84f43b6e90fb None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:20.668726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 187/378] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:20 2022] GET /identity/v3/groups/913b85621ab741538ee206002c163a1e => generated 229 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:20.675572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa55208-32e7-4b3c-b981-84f43b6e90fb None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:20.690470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0d8da-da2c-4ee0-b9ad-bae307f26259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:20.691417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0d8da-da2c-4ee0-b9ad-bae307f26259 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:20.692467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0d8da-da2c-4ee0-b9ad-bae307f26259 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:20.693582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0d8da-da2c-4ee0-b9ad-bae307f26259 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:20.695460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0d8da-da2c-4ee0-b9ad-bae307f26259 None admin] RBAC: Adding query filter params (group.id=00df45b9a8764d11a3d06bc6c30efe31, role.id=9df5a7d7691c49b9a78f0db0a050b7fe, scope.project.id=c9ba9949b27349ab83f686eee37fe6f7) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:20.695989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0d8da-da2c-4ee0-b9ad-bae307f26259 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:20.700570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0d8da-da2c-4ee0-b9ad-bae307f26259 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:20.711223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 192/379] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Aug 18 14:38:20 2022] GET /identity/v3/role_assignments?group.id=00df45b9a8764d11a3d06bc6c30efe31&scope.project.id=c9ba9949b27349ab83f686eee37fe6f7&role.id=9df5a7d7691c49b9a78f0db0a050b7fe => generated 260 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:20.727386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:20.825704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa55208-32e7-4b3c-b981-84f43b6e90fb None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:20.867674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a2a8d-eaf2-49b9-863b-2ec0dea6d5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:20.868659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2a8d-eaf2-49b9-863b-2ec0dea6d5eb None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:20.869160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2a8d-eaf2-49b9-863b-2ec0dea6d5eb None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:20.869623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2a8d-eaf2-49b9-863b-2ec0dea6d5eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:20.870539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a2a8d-eaf2-49b9-863b-2ec0dea6d5eb None admin] RBAC: Adding query filter params (group.id=913b85621ab741538ee206002c163a1e, role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=fa81e49fae72473b99b445418d4fe5de) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:20.871088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a2a8d-eaf2-49b9-863b-2ec0dea6d5eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:20.872723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a2a8d-eaf2-49b9-863b-2ec0dea6d5eb None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:20.879841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 188/380] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Aug 18 14:38:20 2022] GET /identity/v3/role_assignments?group.id=913b85621ab741538ee206002c163a1e&scope.project.id=fa81e49fae72473b99b445418d4fe5de&role.id=9a439cf8eef34ada85428b378878e63e => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:21.008767 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67b637-4c7e-43dd-a28f-ced22f1b8ca0 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:21.009292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67b637-4c7e-43dd-a28f-ced22f1b8ca0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:21.009292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67b637-4c7e-43dd-a28f-ced22f1b8ca0 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:21.010923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 193/381] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:21.016697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1f078-1b87-428a-8425-45fc7edbe395 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:21.017499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1f078-1b87-428a-8425-45fc7edbe395 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:21.017826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1f078-1b87-428a-8425-45fc7edbe395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:21.041144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-97e1f078-1b87-428a-8425-45fc7edbe395 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:21.042371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-97e1f078-1b87-428a-8425-45fc7edbe395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:21.126785 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 189/382] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:21.418417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03725854-b730-4c6a-8c63-9ca1b041f1fa None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:21.419073 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03725854-b730-4c6a-8c63-9ca1b041f1fa None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:21.419570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03725854-b730-4c6a-8c63-9ca1b041f1fa None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:21.421468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 194/383] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:21.427423 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67256d-6a0e-4979-afe3-859b7ec125c1 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:21.427758 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67256d-6a0e-4979-afe3-859b7ec125c1 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:21.428040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67256d-6a0e-4979-afe3-859b7ec125c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:21.453025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-eb67256d-6a0e-4979-afe3-859b7ec125c1 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:21.454768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-eb67256d-6a0e-4979-afe3-859b7ec125c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:21.476031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:21.490569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03725854-b730-4c6a-8c63-9ca1b041f1fa None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:21.553116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 190/384] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:21.622409 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03725854-b730-4c6a-8c63-9ca1b041f1fa None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:21.647918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:21.666830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddf7c0-6ab9-4e54-92df-20ffdcaf35b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:21.667509 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddf7c0-6ab9-4e54-92df-20ffdcaf35b5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:21.667807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddf7c0-6ab9-4e54-92df-20ffdcaf35b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:21.668079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddf7c0-6ab9-4e54-92df-20ffdcaf35b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:21.682903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddf7c0-6ab9-4e54-92df-20ffdcaf35b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-38ddf7c0-6ab9-4e54-92df-20ffdcaf35b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:21.687734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:21.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:21.689505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 195/385] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:38:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:21.694962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67256d-6a0e-4979-afe3-859b7ec125c1 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:21.808829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67256d-6a0e-4979-afe3-859b7ec125c1 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:21.850236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb812db-efd7-429d-80b4-99c29f1c7b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:21.850845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb812db-efd7-429d-80b4-99c29f1c7b0f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:21.851118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb812db-efd7-429d-80b4-99c29f1c7b0f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:21.851386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb812db-efd7-429d-80b4-99c29f1c7b0f None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:21.852085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb812db-efd7-429d-80b4-99c29f1c7b0f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:21.852381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb812db-efd7-429d-80b4-99c29f1c7b0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:21.853940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb812db-efd7-429d-80b4-99c29f1c7b0f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:21.867143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 191/386] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:21.872708 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddf7c0-6ab9-4e54-92df-20ffdcaf35b5 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:21.985823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb812db-efd7-429d-80b4-99c29f1c7b0f None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:22.018735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddf7c0-6ab9-4e54-92df-20ffdcaf35b5 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:22.020090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:22.070793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73940a44-bdc9-4949-bca7-0f815d1b77ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:22.070793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73940a44-bdc9-4949-bca7-0f815d1b77ec None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:22.071055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73940a44-bdc9-4949-bca7-0f815d1b77ec None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:22.071372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73940a44-bdc9-4949-bca7-0f815d1b77ec None admin] PATH_INFO: `/v3/groups/00df45b9a8764d11a3d06bc6c30efe31` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:22.072520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73940a44-bdc9-4949-bca7-0f815d1b77ec None admin] RBAC: Authorizing `identity:get_group(group_id=00df45b9a8764d11a3d06bc6c30efe31)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:22.073987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73940a44-bdc9-4949-bca7-0f815d1b77ec None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:22.076753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 196/387] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:21 2022] GET /identity/v3/groups/00df45b9a8764d11a3d06bc6c30efe31 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:22.088319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73940a44-bdc9-4949-bca7-0f815d1b77ec None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:22.146002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb812db-efd7-429d-80b4-99c29f1c7b0f None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:22.186592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8e9a3-be5c-4357-9a6e-4cf9733ae5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:22.187585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8e9a3-be5c-4357-9a6e-4cf9733ae5d5 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:22.188760 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8e9a3-be5c-4357-9a6e-4cf9733ae5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:22.189224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8e9a3-be5c-4357-9a6e-4cf9733ae5d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:22.200574 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d8e9a3-be5c-4357-9a6e-4cf9733ae5d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:22.201783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-79d8e9a3-be5c-4357-9a6e-4cf9733ae5d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:22.209040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:22.209977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:22.210762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:22.210762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:22.210762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:22.210762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:22.210762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:22.210762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:22.210762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:22.210762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:22.210762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:22.210762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:22.210762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:22.210762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:22.213049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 192/388] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:22.218887 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8e9a3-be5c-4357-9a6e-4cf9733ae5d5 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:22.301100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:22.327828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8e9a3-be5c-4357-9a6e-4cf9733ae5d5 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:22.333957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73940a44-bdc9-4949-bca7-0f815d1b77ec None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:22.363470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380eb7cf-202a-4bb9-adfb-3b03a3d3b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:22.364472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380eb7cf-202a-4bb9-adfb-3b03a3d3b25a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:22.364972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380eb7cf-202a-4bb9-adfb-3b03a3d3b25a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:22.365451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380eb7cf-202a-4bb9-adfb-3b03a3d3b25a None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:22.366303 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380eb7cf-202a-4bb9-adfb-3b03a3d3b25a None admin] RBAC: Adding query filter params (name=member) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:22.366797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380eb7cf-202a-4bb9-adfb-3b03a3d3b25a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:22.368404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380eb7cf-202a-4bb9-adfb-3b03a3d3b25a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:22.377700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 193/389] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:22.383399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380eb7cf-202a-4bb9-adfb-3b03a3d3b25a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:22.440499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cbead-c4b1-46f5-9635-35ce70fd9a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:22.441303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cbead-c4b1-46f5-9635-35ce70fd9a38 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:22.441481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cbead-c4b1-46f5-9635-35ce70fd9a38 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:22.441714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cbead-c4b1-46f5-9635-35ce70fd9a38 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:22.442992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cbead-c4b1-46f5-9635-35ce70fd9a38 None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:22.444531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cbead-c4b1-46f5-9635-35ce70fd9a38 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:22.451735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 197/390] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:22 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:22.460856 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cbead-c4b1-46f5-9635-35ce70fd9a38 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:22.496265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380eb7cf-202a-4bb9-adfb-3b03a3d3b25a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:22.531990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73aa4b0-e1ba-40cd-9742-7e9419a52f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:22.532963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73aa4b0-e1ba-40cd-9742-7e9419a52f81 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:22.533455 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73aa4b0-e1ba-40cd-9742-7e9419a52f81 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:22.533939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73aa4b0-e1ba-40cd-9742-7e9419a52f81 None admin] PATH_INFO: `/v3/groups/00df45b9a8764d11a3d06bc6c30efe31` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:22.535259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73aa4b0-e1ba-40cd-9742-7e9419a52f81 None admin] RBAC: Authorizing `identity:get_group(group_id=00df45b9a8764d11a3d06bc6c30efe31)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:22.537211 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73aa4b0-e1ba-40cd-9742-7e9419a52f81 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:22.539313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 194/391] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:22 2022] GET /identity/v3/groups/00df45b9a8764d11a3d06bc6c30efe31 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:22.545018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73aa4b0-e1ba-40cd-9742-7e9419a52f81 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:22.582592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:22.665589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73aa4b0-e1ba-40cd-9742-7e9419a52f81 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:22.667196 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cbead-c4b1-46f5-9635-35ce70fd9a38 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:22.682137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:22.704175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640446e-b6d8-4b7d-89e8-b3ddc5fd0ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:22.704831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640446e-b6d8-4b7d-89e8-b3ddc5fd0ec6 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:22.705115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640446e-b6d8-4b7d-89e8-b3ddc5fd0ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:22.705424 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640446e-b6d8-4b7d-89e8-b3ddc5fd0ec6 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c/groups/00df45b9a8764d11a3d06bc6c30efe31/roles/9df5a7d7691c49b9a78f0db0a050b7fe` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:22.709370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640446e-b6d8-4b7d-89e8-b3ddc5fd0ec6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c37e716487e9421694a14be7fa5c2d0c, group_id=00df45b9a8764d11a3d06bc6c30efe31, role_id=9df5a7d7691c49b9a78f0db0a050b7fe)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:22.710191 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace9ee2-08ee-4066-b07a-d7fe59d7cc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:22.710827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640446e-b6d8-4b7d-89e8-b3ddc5fd0ec6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:22.711274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace9ee2-08ee-4066-b07a-d7fe59d7cc66 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:22.711779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace9ee2-08ee-4066-b07a-d7fe59d7cc66 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:22.712245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace9ee2-08ee-4066-b07a-d7fe59d7cc66 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:22.713818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace9ee2-08ee-4066-b07a-d7fe59d7cc66 None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:22.716208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace9ee2-08ee-4066-b07a-d7fe59d7cc66 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:22.723848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 195/392] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:22 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:22.725452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed699dc2-d134-4890-8682-2cdde39f2136 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:22.725930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed699dc2-d134-4890-8682-2cdde39f2136 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:22.726444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed699dc2-d134-4890-8682-2cdde39f2136 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:22.727023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 198/393] 10.222.0.36 () {58 vars in 1442 bytes} [Thu Aug 18 14:38:22 2022] PUT /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c/groups/00df45b9a8764d11a3d06bc6c30efe31/roles/9df5a7d7691c49b9a78f0db0a050b7fe => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:22.727925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640446e-b6d8-4b7d-89e8-b3ddc5fd0ec6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:22.728433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 196/394] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:22.738587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d2a3d-38cf-4e91-bc46-7ff48e9bbe14 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:22.739227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d2a3d-38cf-4e91-bc46-7ff48e9bbe14 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:22.739720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d2a3d-38cf-4e91-bc46-7ff48e9bbe14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:22.772761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-dc8d2a3d-38cf-4e91-bc46-7ff48e9bbe14 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:22.774331 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-dc8d2a3d-38cf-4e91-bc46-7ff48e9bbe14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:22.849621 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640446e-b6d8-4b7d-89e8-b3ddc5fd0ec6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:22.875263 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 197/395] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:22.907548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5a222-5f84-4a29-b13c-68c464c6a068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:22.908630 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5a222-5f84-4a29-b13c-68c464c6a068 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:22.909118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5a222-5f84-4a29-b13c-68c464c6a068 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:22.909594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5a222-5f84-4a29-b13c-68c464c6a068 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c/groups/00df45b9a8764d11a3d06bc6c30efe31/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:22.912027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5a222-5f84-4a29-b13c-68c464c6a068 None admin] RBAC: Authorizing `identity:create_grant(project_id=c37e716487e9421694a14be7fa5c2d0c, group_id=00df45b9a8764d11a3d06bc6c30efe31, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:22.913947 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5a222-5f84-4a29-b13c-68c464c6a068 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:22.954237 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 199/396] 10.222.0.36 () {58 vars in 1442 bytes} [Thu Aug 18 14:38:22 2022] PUT /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c/groups/00df45b9a8764d11a3d06bc6c30efe31/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:23.093004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594f9bc-348d-4d43-b3ec-14f9308c17f1 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:23.093970 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594f9bc-348d-4d43-b3ec-14f9308c17f1 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:23.095100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594f9bc-348d-4d43-b3ec-14f9308c17f1 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:23.096921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 198/397] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:23.102942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0fb4a-93f8-439e-b428-a69576905f94 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:23.103873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0fb4a-93f8-439e-b428-a69576905f94 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:23.104359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0fb4a-93f8-439e-b428-a69576905f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:23.126596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-e3a0fb4a-93f8-439e-b428-a69576905f94 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:23.128257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e3a0fb4a-93f8-439e-b428-a69576905f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:23.160052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:23.210698 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594f9bc-348d-4d43-b3ec-14f9308c17f1 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:23.213039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 200/398] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:23.215487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:23.316030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44de7b0-6138-411b-bbe1-24f2f9af3f8b None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:23.316684 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44de7b0-6138-411b-bbe1-24f2f9af3f8b None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:23.317350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44de7b0-6138-411b-bbe1-24f2f9af3f8b None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:23.319779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 201/399] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:23.324606 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91264dfa-8852-4b50-b0c6-764c87f2bce1 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:23.325128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91264dfa-8852-4b50-b0c6-764c87f2bce1 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:23.326064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91264dfa-8852-4b50-b0c6-764c87f2bce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:23.357830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-91264dfa-8852-4b50-b0c6-764c87f2bce1 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:23.359184 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-91264dfa-8852-4b50-b0c6-764c87f2bce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:23.368573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594f9bc-348d-4d43-b3ec-14f9308c17f1 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:23.434864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc7e77-4b32-4929-9bf4-5fa1129b4a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:23.435519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc7e77-4b32-4929-9bf4-5fa1129b4a0e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:23.435823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc7e77-4b32-4929-9bf4-5fa1129b4a0e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:23.436095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc7e77-4b32-4929-9bf4-5fa1129b4a0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:23.456941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 202/400] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:23.472298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc7e77-4b32-4929-9bf4-5fa1129b4a0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-01fc7e77-4b32-4929-9bf4-5fa1129b4a0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:23.477144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:23.478178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:23.479199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 199/401] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:23.484001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91264dfa-8852-4b50-b0c6-764c87f2bce1 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:23.582625 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc7e77-4b32-4929-9bf4-5fa1129b4a0e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:23.633388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:23.657554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91264dfa-8852-4b50-b0c6-764c87f2bce1 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:23.667116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:23.710001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc7e77-4b32-4929-9bf4-5fa1129b4a0e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:23.740723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73702fa-327f-416e-9932-75bdf3dbb414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:23.741503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73702fa-327f-416e-9932-75bdf3dbb414 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:23.741881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73702fa-327f-416e-9932-75bdf3dbb414 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:23.742241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73702fa-327f-416e-9932-75bdf3dbb414 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:23.742901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73702fa-327f-416e-9932-75bdf3dbb414 None admin] RBAC: Adding query filter params (name=member) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:23.743259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73702fa-327f-416e-9932-75bdf3dbb414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:23.748981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73702fa-327f-416e-9932-75bdf3dbb414 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:23.754361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cfb05-e25f-4139-9fde-b5ff514e9ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:23.755353 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cfb05-e25f-4139-9fde-b5ff514e9ffa None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:23.756653 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cfb05-e25f-4139-9fde-b5ff514e9ffa None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:23.759662 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cfb05-e25f-4139-9fde-b5ff514e9ffa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:23.766176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 203/402] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:23.771036 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73702fa-327f-416e-9932-75bdf3dbb414 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:23.778971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1cfb05-e25f-4139-9fde-b5ff514e9ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-1a1cfb05-e25f-4139-9fde-b5ff514e9ffa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:23.782748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:23.783627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:23.784449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 200/403] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:38:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:23.785105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cfb05-e25f-4139-9fde-b5ff514e9ffa None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:23.902084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cfb05-e25f-4139-9fde-b5ff514e9ffa None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:23.902808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73702fa-327f-416e-9932-75bdf3dbb414 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:23.930177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:23.944767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e6b32-3d7a-447b-8ee8-89b27846206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:23.945846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e6b32-3d7a-447b-8ee8-89b27846206d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:23.946303 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e6b32-3d7a-447b-8ee8-89b27846206d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:23.946576 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4f254-aab2-4410-a72c-6b882646adf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:23.946990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e6b32-3d7a-447b-8ee8-89b27846206d None admin] PATH_INFO: `/v3/groups/00df45b9a8764d11a3d06bc6c30efe31` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:23.947195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4f254-aab2-4410-a72c-6b882646adf3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:23.947470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4f254-aab2-4410-a72c-6b882646adf3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:23.947765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4f254-aab2-4410-a72c-6b882646adf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:23.948856 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e6b32-3d7a-447b-8ee8-89b27846206d None admin] RBAC: Authorizing `identity:get_group(group_id=00df45b9a8764d11a3d06bc6c30efe31)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:23.951199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e6b32-3d7a-447b-8ee8-89b27846206d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:23.953332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 201/404] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:23 2022] GET /identity/v3/groups/00df45b9a8764d11a3d06bc6c30efe31 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:23.957998 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e6b32-3d7a-447b-8ee8-89b27846206d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:23.967121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4f254-aab2-4410-a72c-6b882646adf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-f4e4f254-aab2-4410-a72c-6b882646adf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:23.973249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:23.974146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:23.974928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 204/405] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:38:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:23.975580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4f254-aab2-4410-a72c-6b882646adf3 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:23.993769 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:24.083242 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4f254-aab2-4410-a72c-6b882646adf3 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:24.094878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e6b32-3d7a-447b-8ee8-89b27846206d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:24.118737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fe955-d25b-4e35-ac5e-d2a660ab7ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:24.120043 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fe955-d25b-4e35-ac5e-d2a660ab7ef6 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:24.120471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fe955-d25b-4e35-ac5e-d2a660ab7ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:24.120890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fe955-d25b-4e35-ac5e-d2a660ab7ef6 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:24.122175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fe955-d25b-4e35-ac5e-d2a660ab7ef6 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:24.124270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fe955-d25b-4e35-ac5e-d2a660ab7ef6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:24.127065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 205/406] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:23 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:24.128258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fe955-d25b-4e35-ac5e-d2a660ab7ef6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:24.141664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f33655-1c1d-4e2f-bc0e-494feacb6f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:24.142765 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f33655-1c1d-4e2f-bc0e-494feacb6f36 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:24.143144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f33655-1c1d-4e2f-bc0e-494feacb6f36 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:24.143829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f33655-1c1d-4e2f-bc0e-494feacb6f36 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:24.144609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f33655-1c1d-4e2f-bc0e-494feacb6f36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:24.145015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f33655-1c1d-4e2f-bc0e-494feacb6f36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:24.147081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f33655-1c1d-4e2f-bc0e-494feacb6f36 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:24.157909 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 202/407] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:24.160499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f33655-1c1d-4e2f-bc0e-494feacb6f36 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:24.229063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:24.229925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:24.243967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fe955-d25b-4e35-ac5e-d2a660ab7ef6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:24.278302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742eea4-1a7e-4d67-bf3c-57374b6aa3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:24.279063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742eea4-1a7e-4d67-bf3c-57374b6aa3f3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:24.279446 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742eea4-1a7e-4d67-bf3c-57374b6aa3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:24.279855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742eea4-1a7e-4d67-bf3c-57374b6aa3f3 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:24.280709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742eea4-1a7e-4d67-bf3c-57374b6aa3f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:24.281051 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742eea4-1a7e-4d67-bf3c-57374b6aa3f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:24.283102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742eea4-1a7e-4d67-bf3c-57374b6aa3f3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:24.292609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 206/408] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:24.293277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f33655-1c1d-4e2f-bc0e-494feacb6f36 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:24.293811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742eea4-1a7e-4d67-bf3c-57374b6aa3f3 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:24.332914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe6757-c172-4fd3-b243-f583844205c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:24.333823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe6757-c172-4fd3-b243-f583844205c8 None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:24.334288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe6757-c172-4fd3-b243-f583844205c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:24.334706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe6757-c172-4fd3-b243-f583844205c8 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7/groups/00df45b9a8764d11a3d06bc6c30efe31/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:24.337097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe6757-c172-4fd3-b243-f583844205c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9ba9949b27349ab83f686eee37fe6f7, group_id=00df45b9a8764d11a3d06bc6c30efe31, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:24.338973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe6757-c172-4fd3-b243-f583844205c8 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:24.357935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 203/409] 10.222.0.36 () {58 vars in 1442 bytes} [Thu Aug 18 14:38:24 2022] PUT /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7/groups/00df45b9a8764d11a3d06bc6c30efe31/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:24.359345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe6757-c172-4fd3-b243-f583844205c8 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:24.478187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742eea4-1a7e-4d67-bf3c-57374b6aa3f3 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:24.481639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe6757-c172-4fd3-b243-f583844205c8 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:24.496845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:24.521933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193d44a-b3a1-495e-9375-fca244a93475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:24.523017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193d44a-b3a1-495e-9375-fca244a93475 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:24.523502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193d44a-b3a1-495e-9375-fca244a93475 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:24.524004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193d44a-b3a1-495e-9375-fca244a93475 None admin] PATH_INFO: `/v3/groups/913b85621ab741538ee206002c163a1e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:24.525526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193d44a-b3a1-495e-9375-fca244a93475 None admin] RBAC: Authorizing `identity:get_group(group_id=913b85621ab741538ee206002c163a1e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:24.527259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193d44a-b3a1-495e-9375-fca244a93475 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:24.529285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 204/410] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:24 2022] GET /identity/v3/groups/913b85621ab741538ee206002c163a1e => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:24.535494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193d44a-b3a1-495e-9375-fca244a93475 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:24.563968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a15b5-85ff-4428-9471-f24159cdfbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:24.564618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a15b5-85ff-4428-9471-f24159cdfbb0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:24.564894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a15b5-85ff-4428-9471-f24159cdfbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:24.565166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a15b5-85ff-4428-9471-f24159cdfbb0 None admin] PATH_INFO: `/v3/groups/00df45b9a8764d11a3d06bc6c30efe31` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:24.566297 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a15b5-85ff-4428-9471-f24159cdfbb0 None admin] RBAC: Authorizing `identity:get_group(group_id=00df45b9a8764d11a3d06bc6c30efe31)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:24.567754 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a15b5-85ff-4428-9471-f24159cdfbb0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:24.573926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 207/411] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:24 2022] GET /identity/v3/groups/00df45b9a8764d11a3d06bc6c30efe31 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:24.574915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a15b5-85ff-4428-9471-f24159cdfbb0 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:24.593026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:24.672098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193d44a-b3a1-495e-9375-fca244a93475 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:24.694019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a15b5-85ff-4428-9471-f24159cdfbb0 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:24.709621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8219c-6927-4c22-b368-f18cc5dc0443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:24.710690 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8219c-6927-4c22-b368-f18cc5dc0443 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:24.711174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8219c-6927-4c22-b368-f18cc5dc0443 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:24.711799 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8219c-6927-4c22-b368-f18cc5dc0443 None admin] PATH_INFO: `/v3/projects/fa81e49fae72473b99b445418d4fe5de` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:24.713192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8219c-6927-4c22-b368-f18cc5dc0443 None admin] RBAC: Authorizing `identity:get_project(project_id=fa81e49fae72473b99b445418d4fe5de)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:24.715049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8219c-6927-4c22-b368-f18cc5dc0443 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:24.717382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 205/412] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:24 2022] GET /identity/v3/projects/fa81e49fae72473b99b445418d4fe5de => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:24.723236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8219c-6927-4c22-b368-f18cc5dc0443 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:24.729040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429efa24-8eb8-4c9d-babc-c478b63eeff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:24.729619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429efa24-8eb8-4c9d-babc-c478b63eeff5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:24.729889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429efa24-8eb8-4c9d-babc-c478b63eeff5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:24.730164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429efa24-8eb8-4c9d-babc-c478b63eeff5 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:24.731293 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429efa24-8eb8-4c9d-babc-c478b63eeff5 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:24.732753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429efa24-8eb8-4c9d-babc-c478b63eeff5 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:24.734678 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 208/413] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:24 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:24.744665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429efa24-8eb8-4c9d-babc-c478b63eeff5 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:24.800064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:24.836404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:24.856138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429efa24-8eb8-4c9d-babc-c478b63eeff5 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:24.891131 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50297f0a-9021-424c-a031-8ec88f8d0502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:24.891997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50297f0a-9021-424c-a031-8ec88f8d0502 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:24.892429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50297f0a-9021-424c-a031-8ec88f8d0502 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:24.892803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50297f0a-9021-424c-a031-8ec88f8d0502 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7/groups/00df45b9a8764d11a3d06bc6c30efe31/roles/9df5a7d7691c49b9a78f0db0a050b7fe` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:24.897737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50297f0a-9021-424c-a031-8ec88f8d0502 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9ba9949b27349ab83f686eee37fe6f7, group_id=00df45b9a8764d11a3d06bc6c30efe31, role_id=9df5a7d7691c49b9a78f0db0a050b7fe)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:24.899198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50297f0a-9021-424c-a031-8ec88f8d0502 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:24.899306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8219c-6927-4c22-b368-f18cc5dc0443 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:24.934083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 209/414] 10.222.0.36 () {58 vars in 1442 bytes} [Thu Aug 18 14:38:24 2022] PUT /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7/groups/00df45b9a8764d11a3d06bc6c30efe31/roles/9df5a7d7691c49b9a78f0db0a050b7fe => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:24.955325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59086d27-1c3f-4c2e-98d7-7add9a9e56d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:24.956044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086d27-1c3f-4c2e-98d7-7add9a9e56d5 None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:24.956323 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086d27-1c3f-4c2e-98d7-7add9a9e56d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:24.956592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086d27-1c3f-4c2e-98d7-7add9a9e56d5 None admin] PATH_INFO: `/v3/projects/fa81e49fae72473b99b445418d4fe5de/groups/913b85621ab741538ee206002c163a1e/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:24.961196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59086d27-1c3f-4c2e-98d7-7add9a9e56d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa81e49fae72473b99b445418d4fe5de, group_id=913b85621ab741538ee206002c163a1e, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:24.962710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59086d27-1c3f-4c2e-98d7-7add9a9e56d5 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:24.990145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 206/415] 10.222.0.36 () {58 vars in 1442 bytes} [Thu Aug 18 14:38:24 2022] PUT /identity/v3/projects/fa81e49fae72473b99b445418d4fe5de/groups/913b85621ab741538ee206002c163a1e/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:38:25.055113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363ca65-9d7f-4c84-b11e-b00dbdaf9c86 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:25.055436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363ca65-9d7f-4c84-b11e-b00dbdaf9c86 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:25.055671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363ca65-9d7f-4c84-b11e-b00dbdaf9c86 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:25.057338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 210/416] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:25.063568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af04f64-3551-4638-9c75-741f525e3416 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:25.063953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af04f64-3551-4638-9c75-741f525e3416 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:25.064234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af04f64-3551-4638-9c75-741f525e3416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:25.085242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4af04f64-3551-4638-9c75-741f525e3416 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:25.086490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4af04f64-3551-4638-9c75-741f525e3416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:25.163066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 207/417] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:25.228913 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c47148-34cf-4901-8e5a-24adcf1ec3ec None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:25.229416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c47148-34cf-4901-8e5a-24adcf1ec3ec None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:25.229520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c47148-34cf-4901-8e5a-24adcf1ec3ec None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:25.231023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 211/418] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:25.236639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad8908-d176-4552-ad86-99f5825b8589 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:25.236919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad8908-d176-4552-ad86-99f5825b8589 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:25.237189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad8908-d176-4552-ad86-99f5825b8589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:25.256462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-b7ad8908-d176-4552-ad86-99f5825b8589 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:25.257573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b7ad8908-d176-4552-ad86-99f5825b8589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:25.287441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:25.336390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 208/419] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:25.483021 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c47148-34cf-4901-8e5a-24adcf1ec3ec None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:25.609476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad8908-d176-4552-ad86-99f5825b8589 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:25.622818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c47148-34cf-4901-8e5a-24adcf1ec3ec None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:25.663912 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbffa82-70fc-4ccb-bae0-1d8ba284a845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:25.664609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbffa82-70fc-4ccb-bae0-1d8ba284a845 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:25.665044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbffa82-70fc-4ccb-bae0-1d8ba284a845 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:25.665330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbffa82-70fc-4ccb-bae0-1d8ba284a845 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:25.677878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbffa82-70fc-4ccb-bae0-1d8ba284a845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-fdbffa82-70fc-4ccb-bae0-1d8ba284a845 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:25.683145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:25.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:25.687153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 212/420] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:38:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:25.692911 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbffa82-70fc-4ccb-bae0-1d8ba284a845 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:25.696986 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:25.815946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad8908-d176-4552-ad86-99f5825b8589 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:25.864207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbffa82-70fc-4ccb-bae0-1d8ba284a845 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:25.884661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e776330-f8f2-4a21-9fd6-30341303ad7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:25.885298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776330-f8f2-4a21-9fd6-30341303ad7a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:25.885548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776330-f8f2-4a21-9fd6-30341303ad7a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:25.885822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776330-f8f2-4a21-9fd6-30341303ad7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:25.907910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e776330-f8f2-4a21-9fd6-30341303ad7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-4e776330-f8f2-4a21-9fd6-30341303ad7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:25.912608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:25.914664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:25.916635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 209/421] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:25.925324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e776330-f8f2-4a21-9fd6-30341303ad7a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:25.934620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e5883-c410-462c-9a26-1b360e47aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:25.935226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e5883-c410-462c-9a26-1b360e47aa0a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:25.935497 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e5883-c410-462c-9a26-1b360e47aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:25.935795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e5883-c410-462c-9a26-1b360e47aa0a None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:25.936467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e5883-c410-462c-9a26-1b360e47aa0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:25.936766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e5883-c410-462c-9a26-1b360e47aa0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:25.938790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e5883-c410-462c-9a26-1b360e47aa0a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:25.957473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 213/422] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:25.964213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e5883-c410-462c-9a26-1b360e47aa0a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:25.981132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:26.071545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:26.105816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e776330-f8f2-4a21-9fd6-30341303ad7a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:26.140454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e5883-c410-462c-9a26-1b360e47aa0a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:26.162260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279ab4e-95a4-4fa4-ab5a-1ef61ac032a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:26.162878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279ab4e-95a4-4fa4-ab5a-1ef61ac032a2 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:26.163151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279ab4e-95a4-4fa4-ab5a-1ef61ac032a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:26.163773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279ab4e-95a4-4fa4-ab5a-1ef61ac032a2 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:26.164102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279ab4e-95a4-4fa4-ab5a-1ef61ac032a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:26.164398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279ab4e-95a4-4fa4-ab5a-1ef61ac032a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:26.165780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279ab4e-95a4-4fa4-ab5a-1ef61ac032a2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:26.187175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 210/423] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:26.189508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279ab4e-95a4-4fa4-ab5a-1ef61ac032a2 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:26.209259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052784f-d83b-439b-b918-ed2f9afcbf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:26.209867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052784f-d83b-439b-b918-ed2f9afcbf96 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:26.210141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052784f-d83b-439b-b918-ed2f9afcbf96 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:26.210409 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052784f-d83b-439b-b918-ed2f9afcbf96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:26.315832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0052784f-d83b-439b-b918-ed2f9afcbf96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:26.317272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0052784f-d83b-439b-b918-ed2f9afcbf96 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:26.366645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:26.385872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279ab4e-95a4-4fa4-ab5a-1ef61ac032a2 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:26.436044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:26.449170 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5e35e-eb01-4922-95c5-c32a9d298ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:26.449787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5e35e-eb01-4922-95c5-c32a9d298ba3 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:26.450064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5e35e-eb01-4922-95c5-c32a9d298ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:26.450331 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5e35e-eb01-4922-95c5-c32a9d298ba3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:26.502371 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 214/424] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:26.508095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052784f-d83b-439b-b918-ed2f9afcbf96 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:26.554710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5e35e-eb01-4922-95c5-c32a9d298ba3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:26.556227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5e35e-eb01-4922-95c5-c32a9d298ba3 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:26.660473 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 211/425] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:26.664080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:26.674741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5e35e-eb01-4922-95c5-c32a9d298ba3 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:26.745324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052784f-d83b-439b-b918-ed2f9afcbf96 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:26.781430 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7813886-9b2a-42f4-b271-581d50437812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:26.782184 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7813886-9b2a-42f4-b271-581d50437812 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:26.782461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7813886-9b2a-42f4-b271-581d50437812 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:26.782728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7813886-9b2a-42f4-b271-581d50437812 None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:26.783869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7813886-9b2a-42f4-b271-581d50437812 None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:26.785307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7813886-9b2a-42f4-b271-581d50437812 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:26.787202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 215/426] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:26 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:26.788403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5225393-4925-4001-8bda-0699323b52e3 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:26.788691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5225393-4925-4001-8bda-0699323b52e3 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:26.788960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5225393-4925-4001-8bda-0699323b52e3 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:26.794182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 216/427] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:26.794182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5225393-4925-4001-8bda-0699323b52e3 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:26.824171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:26.833606 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5e35e-eb01-4922-95c5-c32a9d298ba3 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:26.886948 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d05fc-8638-4218-a5a5-877f2244684e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:26.888246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d05fc-8638-4218-a5a5-877f2244684e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:26.888991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d05fc-8638-4218-a5a5-877f2244684e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:26.889510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d05fc-8638-4218-a5a5-877f2244684e None admin] PATH_INFO: `/v3/projects/c37e716487e9421694a14be7fa5c2d0c` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:26.891328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d05fc-8638-4218-a5a5-877f2244684e None admin] RBAC: Authorizing `identity:get_project(project_id=c37e716487e9421694a14be7fa5c2d0c)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:26.893094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d05fc-8638-4218-a5a5-877f2244684e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:26.895025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 212/428] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:26 2022] GET /identity/v3/projects/c37e716487e9421694a14be7fa5c2d0c => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:26.896623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01686a-0217-42d5-a275-5aa8b0850183 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:26.897165 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01686a-0217-42d5-a275-5aa8b0850183 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:26.897638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01686a-0217-42d5-a275-5aa8b0850183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:26.916378 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5225393-4925-4001-8bda-0699323b52e3 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:26.923536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4b01686a-0217-42d5-a275-5aa8b0850183 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:26.925017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4b01686a-0217-42d5-a275-5aa8b0850183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:26.956813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2012ab7-3434-49f6-900d-5fcd104157c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:26.957441 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2012ab7-3434-49f6-900d-5fcd104157c3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:26.957714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2012ab7-3434-49f6-900d-5fcd104157c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:26.957982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2012ab7-3434-49f6-900d-5fcd104157c3 None admin] PATH_INFO: `/v3/groups/00df45b9a8764d11a3d06bc6c30efe31` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:26.959092 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2012ab7-3434-49f6-900d-5fcd104157c3 None admin] RBAC: Authorizing `identity:get_group(group_id=00df45b9a8764d11a3d06bc6c30efe31)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:26.960661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2012ab7-3434-49f6-900d-5fcd104157c3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:26.962397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 217/429] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:26 2022] GET /identity/v3/groups/00df45b9a8764d11a3d06bc6c30efe31 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:26.963313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2012ab7-3434-49f6-900d-5fcd104157c3 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:27.006696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:27.028305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 213/430] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:27.029556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01686a-0217-42d5-a275-5aa8b0850183 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:27.113036 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:27.118078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2012ab7-3434-49f6-900d-5fcd104157c3 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:27.153020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01686a-0217-42d5-a275-5aa8b0850183 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:27.171550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0da030-e5cf-47b7-8da3-6f063e918480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:27.172270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0da030-e5cf-47b7-8da3-6f063e918480 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:27.172708 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0da030-e5cf-47b7-8da3-6f063e918480 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:27.172991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0da030-e5cf-47b7-8da3-6f063e918480 None admin] PATH_INFO: `/v3/groups/00df45b9a8764d11a3d06bc6c30efe31` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:27.174492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0da030-e5cf-47b7-8da3-6f063e918480 None admin] RBAC: Authorizing `identity:get_group(group_id=00df45b9a8764d11a3d06bc6c30efe31)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:27.176230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0da030-e5cf-47b7-8da3-6f063e918480 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:27.178455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 218/431] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:26 2022] GET /identity/v3/groups/00df45b9a8764d11a3d06bc6c30efe31 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:27.179461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96759a5-5664-493b-b76d-3ea6012ecb5a None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:27.179847 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96759a5-5664-493b-b76d-3ea6012ecb5a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:27.180123 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96759a5-5664-493b-b76d-3ea6012ecb5a None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:27.183812 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 219/432] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:27.184775 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96759a5-5664-493b-b76d-3ea6012ecb5a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:27.207020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12f929-0917-4504-bfbb-00440543a46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:27.208557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12f929-0917-4504-bfbb-00440543a46c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:27.209430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12f929-0917-4504-bfbb-00440543a46c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:27.209910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12f929-0917-4504-bfbb-00440543a46c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:27.210837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12f929-0917-4504-bfbb-00440543a46c None admin] RBAC: Adding query filter params (group.id=00df45b9a8764d11a3d06bc6c30efe31, role.id=9df5a7d7691c49b9a78f0db0a050b7fe, scope.project.id=c37e716487e9421694a14be7fa5c2d0c) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:27.211443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12f929-0917-4504-bfbb-00440543a46c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:27.213941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12f929-0917-4504-bfbb-00440543a46c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:27.227752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 214/433] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Aug 18 14:38:27 2022] GET /identity/v3/role_assignments?group.id=00df45b9a8764d11a3d06bc6c30efe31&scope.project.id=c37e716487e9421694a14be7fa5c2d0c&role.id=9df5a7d7691c49b9a78f0db0a050b7fe => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:27.227752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca639c0-c314-441a-bbea-6eb86191a723 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:27.227752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca639c0-c314-441a-bbea-6eb86191a723 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:27.227752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca639c0-c314-441a-bbea-6eb86191a723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:27.262869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-6ca639c0-c314-441a-bbea-6eb86191a723 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:27.264250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6ca639c0-c314-441a-bbea-6eb86191a723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:27.359471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:27.366491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96759a5-5664-493b-b76d-3ea6012ecb5a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:27.387938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 215/434] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:27.388852 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca639c0-c314-441a-bbea-6eb86191a723 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:27.423238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fdf6f-1b96-4999-a8fc-0128136f78c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:27.424245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fdf6f-1b96-4999-a8fc-0128136f78c1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:27.424733 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fdf6f-1b96-4999-a8fc-0128136f78c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:27.425257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fdf6f-1b96-4999-a8fc-0128136f78c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:27.426426 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fdf6f-1b96-4999-a8fc-0128136f78c1 None admin] RBAC: Adding query filter params (group.id=00df45b9a8764d11a3d06bc6c30efe31, role.id=603573d2738b49c389ad913208d2ced1, scope.project.id=c37e716487e9421694a14be7fa5c2d0c) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:27.426940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fdf6f-1b96-4999-a8fc-0128136f78c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:27.428695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fdf6f-1b96-4999-a8fc-0128136f78c1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:27.436122 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 220/435] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Aug 18 14:38:27 2022] GET /identity/v3/role_assignments?group.id=00df45b9a8764d11a3d06bc6c30efe31&scope.project.id=c37e716487e9421694a14be7fa5c2d0c&role.id=603573d2738b49c389ad913208d2ced1 => generated 611 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:27.485490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534aacf-bdad-498f-8058-e08681dae326 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:27.485858 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534aacf-bdad-498f-8058-e08681dae326 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:27.486037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534aacf-bdad-498f-8058-e08681dae326 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:27.487375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 221/436] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:38:27.494127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1140104-277f-4133-9f32-506001ca7d89 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:27.494424 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1140104-277f-4133-9f32-506001ca7d89 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:27.494668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1140104-277f-4133-9f32-506001ca7d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:27.522201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f1140104-277f-4133-9f32-506001ca7d89 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:27.523378 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f1140104-277f-4133-9f32-506001ca7d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:27.532854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca639c0-c314-441a-bbea-6eb86191a723 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:27.556460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:27.572133 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026daa8-fbe0-4258-8db0-7f2ecfe6d3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:27.573112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026daa8-fbe0-4258-8db0-7f2ecfe6d3af None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:27.573535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026daa8-fbe0-4258-8db0-7f2ecfe6d3af None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:27.573953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026daa8-fbe0-4258-8db0-7f2ecfe6d3af None admin] PATH_INFO: `/v3/roles/member` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:27.586235 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026daa8-fbe0-4258-8db0-7f2ecfe6d3af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-3026daa8-fbe0-4258-8db0-7f2ecfe6d3af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:27.591404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:27.592542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:27.593530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:27.593530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:27.593530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:27.593530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:27.593530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:27.593530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:27.593530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:27.593530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:27.593530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:27.593530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:27.593530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 14:38:27.593530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:27.596107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 216/437] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:27.603137 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026daa8-fbe0-4258-8db0-7f2ecfe6d3af None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:27.620787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 222/438] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:27.652404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:27.680112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1140104-277f-4133-9f32-506001ca7d89 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:27.717017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026daa8-fbe0-4258-8db0-7f2ecfe6d3af None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:27.765594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdc5c0-cfeb-4d9f-b252-42bd96b71f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:27.766229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdc5c0-cfeb-4d9f-b252-42bd96b71f37 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:27.766520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdc5c0-cfeb-4d9f-b252-42bd96b71f37 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:27.766813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdc5c0-cfeb-4d9f-b252-42bd96b71f37 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:27.767427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdc5c0-cfeb-4d9f-b252-42bd96b71f37 None admin] RBAC: Adding query filter params (name=member) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:27.767804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdc5c0-cfeb-4d9f-b252-42bd96b71f37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:27.769507 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdc5c0-cfeb-4d9f-b252-42bd96b71f37 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:27.778765 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 217/439] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:38:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:27.786095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdc5c0-cfeb-4d9f-b252-42bd96b71f37 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:27.810120 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1140104-277f-4133-9f32-506001ca7d89 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:27.856792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7273a2-bf0e-4374-8885-1fafc2ce92d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:27.857777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7273a2-bf0e-4374-8885-1fafc2ce92d7 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:27.858277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7273a2-bf0e-4374-8885-1fafc2ce92d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:27.858741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7273a2-bf0e-4374-8885-1fafc2ce92d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:27.873473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7273a2-bf0e-4374-8885-1fafc2ce92d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-cf7273a2-bf0e-4374-8885-1fafc2ce92d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:27.879024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:27.879950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:27.880758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:27.880758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:27.880758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:27.880758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:27.880758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:27.880758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:27.880758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:27.880758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:27.880758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:27.880758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:27.880758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:27.880758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:27.884531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 223/440] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:38:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:27.887510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7273a2-bf0e-4374-8885-1fafc2ce92d7 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:27.916554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdc5c0-cfeb-4d9f-b252-42bd96b71f37 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:27.919029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:27.924041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:27.959645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03be0d-083c-4051-9a59-a2f1891be81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:27.960277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03be0d-083c-4051-9a59-a2f1891be81a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:27.960549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03be0d-083c-4051-9a59-a2f1891be81a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:27.960816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03be0d-083c-4051-9a59-a2f1891be81a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:28.019286 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7273a2-bf0e-4374-8885-1fafc2ce92d7 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:28.042723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03be0d-083c-4051-9a59-a2f1891be81a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:28.044165 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03be0d-083c-4051-9a59-a2f1891be81a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:28.058187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d9811-69ad-49fe-8e55-01f0aa9d7adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:28.058793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d9811-69ad-49fe-8e55-01f0aa9d7adc None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:28.059068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d9811-69ad-49fe-8e55-01f0aa9d7adc None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:28.059333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d9811-69ad-49fe-8e55-01f0aa9d7adc None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:28.060045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d9811-69ad-49fe-8e55-01f0aa9d7adc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:28.060344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d9811-69ad-49fe-8e55-01f0aa9d7adc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:28.061923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d9811-69ad-49fe-8e55-01f0aa9d7adc None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:28.071160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 224/441] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:28.072015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d9811-69ad-49fe-8e55-01f0aa9d7adc None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:28.152771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 218/442] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:28.153855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03be0d-083c-4051-9a59-a2f1891be81a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:28.156863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:28.164631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:28.181186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d9811-69ad-49fe-8e55-01f0aa9d7adc None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:28.214921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ddb0d-c96a-4a25-85c3-18cf40cea6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:28.215537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ddb0d-c96a-4a25-85c3-18cf40cea6f5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:28.215849 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ddb0d-c96a-4a25-85c3-18cf40cea6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:28.216118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ddb0d-c96a-4a25-85c3-18cf40cea6f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:28.226545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ddb0d-c96a-4a25-85c3-18cf40cea6f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-1f3ddb0d-c96a-4a25-85c3-18cf40cea6f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:28.231734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:28.232710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:28.233548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 225/443] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:38:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:28.234198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ddb0d-c96a-4a25-85c3-18cf40cea6f5 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:28.254191 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03be0d-083c-4051-9a59-a2f1891be81a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:28.287555 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e890c-d90f-47a3-9432-16d7a7ddeda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:28.288248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e890c-d90f-47a3-9432-16d7a7ddeda0 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:28.288522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e890c-d90f-47a3-9432-16d7a7ddeda0 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:28.288787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e890c-d90f-47a3-9432-16d7a7ddeda0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:28.333041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ddb0d-c96a-4a25-85c3-18cf40cea6f5 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:28.355392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e890c-d90f-47a3-9432-16d7a7ddeda0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:28.356881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e890c-d90f-47a3-9432-16d7a7ddeda0 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:28.359436 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:28.366534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40067df1-66b7-4d49-a80b-4ccf4484f18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:28.367517 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067df1-66b7-4d49-a80b-4ccf4484f18b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:28.367834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067df1-66b7-4d49-a80b-4ccf4484f18b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:28.368112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067df1-66b7-4d49-a80b-4ccf4484f18b None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:28.369255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40067df1-66b7-4d49-a80b-4ccf4484f18b None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:28.370689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40067df1-66b7-4d49-a80b-4ccf4484f18b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:28.372342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 226/444] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:28 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:28.373101 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40067df1-66b7-4d49-a80b-4ccf4484f18b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:28.387752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:28.457647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 219/445] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:28.458486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e890c-d90f-47a3-9432-16d7a7ddeda0 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:28.470372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40067df1-66b7-4d49-a80b-4ccf4484f18b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:28.504307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690db6b-4206-4988-a1fa-edff4ac92a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:28.504902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690db6b-4206-4988-a1fa-edff4ac92a73 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:28.505180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690db6b-4206-4988-a1fa-edff4ac92a73 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:28.505453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690db6b-4206-4988-a1fa-edff4ac92a73 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:28.506173 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690db6b-4206-4988-a1fa-edff4ac92a73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:28.506472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690db6b-4206-4988-a1fa-edff4ac92a73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:28.507892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690db6b-4206-4988-a1fa-edff4ac92a73 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:28.515810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 227/446] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:28.516597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690db6b-4206-4988-a1fa-edff4ac92a73 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:28.554624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e890c-d90f-47a3-9432-16d7a7ddeda0 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:28.557645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:28.589324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef88fd2-a124-44f4-8a8a-e6e5a17b43dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:28.589875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef88fd2-a124-44f4-8a8a-e6e5a17b43dd None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:28.590148 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef88fd2-a124-44f4-8a8a-e6e5a17b43dd None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:28.590417 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef88fd2-a124-44f4-8a8a-e6e5a17b43dd None admin] PATH_INFO: `/v3/groups/00df45b9a8764d11a3d06bc6c30efe31` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:28.591513 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef88fd2-a124-44f4-8a8a-e6e5a17b43dd None admin] RBAC: Authorizing `identity:get_group(group_id=00df45b9a8764d11a3d06bc6c30efe31)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:28.592988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef88fd2-a124-44f4-8a8a-e6e5a17b43dd None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:28.593110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:28.594596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 220/447] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:28 2022] GET /identity/v3/groups/00df45b9a8764d11a3d06bc6c30efe31 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:28.595363 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef88fd2-a124-44f4-8a8a-e6e5a17b43dd None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:28.618989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690db6b-4206-4988-a1fa-edff4ac92a73 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:28.652037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b1360-ad82-4609-b21e-2a3232460da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:28.652662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1360-ad82-4609-b21e-2a3232460da6 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:28.652941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1360-ad82-4609-b21e-2a3232460da6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:28.653222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1360-ad82-4609-b21e-2a3232460da6 None admin] PATH_INFO: `/v3/projects/c9ba9949b27349ab83f686eee37fe6f7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:28.654341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b1360-ad82-4609-b21e-2a3232460da6 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ba9949b27349ab83f686eee37fe6f7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:28.655801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b1360-ad82-4609-b21e-2a3232460da6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:28.657934 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 228/448] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:28 2022] GET /identity/v3/projects/c9ba9949b27349ab83f686eee37fe6f7 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:28.658700 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b1360-ad82-4609-b21e-2a3232460da6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:28.696189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef88fd2-a124-44f4-8a8a-e6e5a17b43dd None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:28.731814 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aedcf6-54bc-4a40-95a1-0f59cfd43fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:28.732415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aedcf6-54bc-4a40-95a1-0f59cfd43fd0 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:28.732692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aedcf6-54bc-4a40-95a1-0f59cfd43fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:28.732965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aedcf6-54bc-4a40-95a1-0f59cfd43fd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:28.755911 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b1360-ad82-4609-b21e-2a3232460da6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:28.772876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:28.789277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31204ac5-3b36-403e-933f-0d62b38dff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:28.789878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31204ac5-3b36-403e-933f-0d62b38dff24 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:28.790187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31204ac5-3b36-403e-933f-0d62b38dff24 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:28.790431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31204ac5-3b36-403e-933f-0d62b38dff24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:28.791155 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31204ac5-3b36-403e-933f-0d62b38dff24 None admin] RBAC: Adding query filter params (group.id=00df45b9a8764d11a3d06bc6c30efe31, role.id=603573d2738b49c389ad913208d2ced1, scope.project.id=c9ba9949b27349ab83f686eee37fe6f7) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:28.791462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31204ac5-3b36-403e-933f-0d62b38dff24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:28.792950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31204ac5-3b36-403e-933f-0d62b38dff24 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:28.798546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 229/449] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Aug 18 14:38:28 2022] GET /identity/v3/role_assignments?group.id=00df45b9a8764d11a3d06bc6c30efe31&scope.project.id=c9ba9949b27349ab83f686eee37fe6f7&role.id=603573d2738b49c389ad913208d2ced1 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:28.799401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31204ac5-3b36-403e-933f-0d62b38dff24 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:28.800803 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aedcf6-54bc-4a40-95a1-0f59cfd43fd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:28.802593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aedcf6-54bc-4a40-95a1-0f59cfd43fd0 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:28.808729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:28.913826 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31204ac5-3b36-403e-933f-0d62b38dff24 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:28.932615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 221/450] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:28.934070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aedcf6-54bc-4a40-95a1-0f59cfd43fd0 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:28.966256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d79dd-822c-4e09-bf20-09d0125d9929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:28.966904 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d79dd-822c-4e09-bf20-09d0125d9929 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:28.967189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d79dd-822c-4e09-bf20-09d0125d9929 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:28.967465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d79dd-822c-4e09-bf20-09d0125d9929 None admin] PATH_INFO: `/v3/groups/00df45b9a8764d11a3d06bc6c30efe31` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:28.968671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d79dd-822c-4e09-bf20-09d0125d9929 None admin] RBAC: Authorizing `identity:get_group(group_id=00df45b9a8764d11a3d06bc6c30efe31)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:28.970187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d79dd-822c-4e09-bf20-09d0125d9929 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:28.971798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 230/451] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:28 2022] GET /identity/v3/groups/00df45b9a8764d11a3d06bc6c30efe31 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:28.978214 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d79dd-822c-4e09-bf20-09d0125d9929 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:29.024752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:29.061560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aedcf6-54bc-4a40-95a1-0f59cfd43fd0 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:29.066791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:29.094254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d79dd-822c-4e09-bf20-09d0125d9929 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:29.103537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abeb49-cfac-4f67-ab7b-503710e9be73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:29.104376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abeb49-cfac-4f67-ab7b-503710e9be73 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:29.104807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abeb49-cfac-4f67-ab7b-503710e9be73 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:29.105222 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abeb49-cfac-4f67-ab7b-503710e9be73 None admin] PATH_INFO: `/v3/projects/fa81e49fae72473b99b445418d4fe5de` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:29.106504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abeb49-cfac-4f67-ab7b-503710e9be73 None admin] RBAC: Authorizing `identity:get_project(project_id=fa81e49fae72473b99b445418d4fe5de)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:29.107970 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abeb49-cfac-4f67-ab7b-503710e9be73 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:29.110169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 222/452] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:38:28 2022] GET /identity/v3/projects/fa81e49fae72473b99b445418d4fe5de => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:29.117195 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abeb49-cfac-4f67-ab7b-503710e9be73 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:29.142760 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efac6b-e950-4ffe-90ec-c4c7120b574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:29.143374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efac6b-e950-4ffe-90ec-c4c7120b574f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:29.143655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efac6b-e950-4ffe-90ec-c4c7120b574f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:29.143963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efac6b-e950-4ffe-90ec-c4c7120b574f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:29.144697 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efac6b-e950-4ffe-90ec-c4c7120b574f None admin] RBAC: Adding query filter params (group.id=00df45b9a8764d11a3d06bc6c30efe31, role.id=9df5a7d7691c49b9a78f0db0a050b7fe, scope.project.id=c9ba9949b27349ab83f686eee37fe6f7) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:29.145006 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efac6b-e950-4ffe-90ec-c4c7120b574f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:29.146446 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efac6b-e950-4ffe-90ec-c4c7120b574f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:29.153629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 231/453] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Aug 18 14:38:28 2022] GET /identity/v3/role_assignments?group.id=00df45b9a8764d11a3d06bc6c30efe31&scope.project.id=c9ba9949b27349ab83f686eee37fe6f7&role.id=9df5a7d7691c49b9a78f0db0a050b7fe => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:29.227198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abeb49-cfac-4f67-ab7b-503710e9be73 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:29.263237 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb50d2f-43fb-4eba-8f20-18ee9a969994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:29.263960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb50d2f-43fb-4eba-8f20-18ee9a969994 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:29.264156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb50d2f-43fb-4eba-8f20-18ee9a969994 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:29.264421 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb50d2f-43fb-4eba-8f20-18ee9a969994 None admin] PATH_INFO: `/v3/groups/913b85621ab741538ee206002c163a1e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:29.265555 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb50d2f-43fb-4eba-8f20-18ee9a969994 None admin] RBAC: Authorizing `identity:get_group(group_id=913b85621ab741538ee206002c163a1e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:29.267065 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb50d2f-43fb-4eba-8f20-18ee9a969994 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:29.268691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 223/454] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Aug 18 14:38:29 2022] GET /identity/v3/groups/913b85621ab741538ee206002c163a1e => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:29.274674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efac6b-e950-4ffe-90ec-c4c7120b574f None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:29.376248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efac6b-e950-4ffe-90ec-c4c7120b574f None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:29.409426 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c139f-3880-4cc8-939e-6110baef3873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:29.409959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c139f-3880-4cc8-939e-6110baef3873 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:29.410241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c139f-3880-4cc8-939e-6110baef3873 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:29.410520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c139f-3880-4cc8-939e-6110baef3873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:29.411240 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c139f-3880-4cc8-939e-6110baef3873 None admin] RBAC: Adding query filter params (group.id=913b85621ab741538ee206002c163a1e, role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=fa81e49fae72473b99b445418d4fe5de) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:29.411548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c139f-3880-4cc8-939e-6110baef3873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:29.413024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c139f-3880-4cc8-939e-6110baef3873 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:29.418788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 232/455] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Aug 18 14:38:29 2022] GET /identity/v3/role_assignments?group.id=913b85621ab741538ee206002c163a1e&scope.project.id=fa81e49fae72473b99b445418d4fe5de&role.id=9a439cf8eef34ada85428b378878e63e => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:31.777105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e24fd4-3156-4bc8-875b-c0c79d35d9e3 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:31.777622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e24fd4-3156-4bc8-875b-c0c79d35d9e3 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:31.777622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e24fd4-3156-4bc8-875b-c0c79d35d9e3 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:31.779218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 224/456] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:31.784239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b57d48-9897-4e33-9d1f-faf518e2c907 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:31.785062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b57d48-9897-4e33-9d1f-faf518e2c907 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:31.785553 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b57d48-9897-4e33-9d1f-faf518e2c907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:31.794483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:31.811423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f2b57d48-9897-4e33-9d1f-faf518e2c907 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:31.812642 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f2b57d48-9897-4e33-9d1f-faf518e2c907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:31.877023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f060d-904d-4401-9b2f-3f2444ec017c None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:31.877532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f060d-904d-4401-9b2f-3f2444ec017c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:31.877949 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f060d-904d-4401-9b2f-3f2444ec017c None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:31.883762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 225/457] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:31.892891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19eeee3-44f5-4d9e-8092-6f8929f00e0a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:31.898731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19eeee3-44f5-4d9e-8092-6f8929f00e0a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:31.899247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19eeee3-44f5-4d9e-8092-6f8929f00e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:31.901285 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 233/458] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:31.944670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-b19eeee3-44f5-4d9e-8092-6f8929f00e0a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:31.946299 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b19eeee3-44f5-4d9e-8092-6f8929f00e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:31.950209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:32.049302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 226/459] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:32.176055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e20d8-e4c5-4ceb-afe2-1ec6b1de9a1e None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:32.176394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e20d8-e4c5-4ceb-afe2-1ec6b1de9a1e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:32.176608 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e20d8-e4c5-4ceb-afe2-1ec6b1de9a1e None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:32.178079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 234/460] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:32.187183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19eeee3-44f5-4d9e-8092-6f8929f00e0a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:32.191001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3fb17-e98e-47ca-85e2-6124fa6c3054 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:32.191281 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3fb17-e98e-47ca-85e2-6124fa6c3054 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:32.191823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3fb17-e98e-47ca-85e2-6124fa6c3054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:32.219031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-84b3fb17-e98e-47ca-85e2-6124fa6c3054 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:32.221310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-84b3fb17-e98e-47ca-85e2-6124fa6c3054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:32.359189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 235/461] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:32.385662 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19eeee3-44f5-4d9e-8092-6f8929f00e0a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:32.434195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3fb17-e98e-47ca-85e2-6124fa6c3054 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:32.442157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d91e8-7056-4303-b4f1-029d38ade137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:32.442789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d91e8-7056-4303-b4f1-029d38ade137 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:32.443064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d91e8-7056-4303-b4f1-029d38ade137 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:32.443357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d91e8-7056-4303-b4f1-029d38ade137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:32.456064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:32.463999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d91e8-7056-4303-b4f1-029d38ade137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:32.466760 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d91e8-7056-4303-b4f1-029d38ade137 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-5c8d91e8-7056-4303-b4f1-029d38ade137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:32.482553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:32.483617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:32.484568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:32.484568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:32.484568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:32.486085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 227/462] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:32.492039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d91e8-7056-4303-b4f1-029d38ade137 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:32.531539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:32.590826 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3fb17-e98e-47ca-85e2-6124fa6c3054 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:32.638980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe00564-e1ee-4cd5-b986-df52693d4bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:32.639603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d91e8-7056-4303-b4f1-029d38ade137 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:32.640175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe00564-e1ee-4cd5-b986-df52693d4bf3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:32.640448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe00564-e1ee-4cd5-b986-df52693d4bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:32.640720 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe00564-e1ee-4cd5-b986-df52693d4bf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:32.654784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe00564-e1ee-4cd5-b986-df52693d4bf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:32.656799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe00564-e1ee-4cd5-b986-df52693d4bf3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-ebe00564-e1ee-4cd5-b986-df52693d4bf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:32.678906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:32.680100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:32.681189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:32.681189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:32.681189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:32.681189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 236/463] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:32.687335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5c222-a947-4c2e-8288-7a06d749a57c None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:32.687335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5c222-a947-4c2e-8288-7a06d749a57c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:32.687335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5c222-a947-4c2e-8288-7a06d749a57c None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:32.687335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 237/464] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:32.689263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5c222-a947-4c2e-8288-7a06d749a57c None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:32.690961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95c1f9-bc87-41c5-9f1a-7f6bda85f53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:32.690961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95c1f9-bc87-41c5-9f1a-7f6bda85f53e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:32.690961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95c1f9-bc87-41c5-9f1a-7f6bda85f53e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:32.690961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95c1f9-bc87-41c5-9f1a-7f6bda85f53e None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:32.691890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95c1f9-bc87-41c5-9f1a-7f6bda85f53e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:32.691890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95c1f9-bc87-41c5-9f1a-7f6bda85f53e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:32.693093 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95c1f9-bc87-41c5-9f1a-7f6bda85f53e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:32.706071 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 228/465] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:32.711465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95c1f9-bc87-41c5-9f1a-7f6bda85f53e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:32.765288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:32.806649 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5c222-a947-4c2e-8288-7a06d749a57c None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:32.821469 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95c1f9-bc87-41c5-9f1a-7f6bda85f53e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:32.822334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:32.845239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f0b7e-ee6c-4cf6-a9b0-c7fa5a75973a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:32.845901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f0b7e-ee6c-4cf6-a9b0-c7fa5a75973a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:32.846174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f0b7e-ee6c-4cf6-a9b0-c7fa5a75973a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:32.846443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f0b7e-ee6c-4cf6-a9b0-c7fa5a75973a None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:32.847133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f0b7e-ee6c-4cf6-a9b0-c7fa5a75973a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:32.847434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f0b7e-ee6c-4cf6-a9b0-c7fa5a75973a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:32.848877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f0b7e-ee6c-4cf6-a9b0-c7fa5a75973a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:32.866424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ceb38-4e03-4ac9-981a-b40a5fb88ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:32.866618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 238/466] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:32.866728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ceb38-4e03-4ac9-981a-b40a5fb88ba2 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:32.866728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ceb38-4e03-4ac9-981a-b40a5fb88ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:32.866728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ceb38-4e03-4ac9-981a-b40a5fb88ba2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:32.866930 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10bfe2-ebbc-4977-9aae-2d4ef98a7de3 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:32.866930 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10bfe2-ebbc-4977-9aae-2d4ef98a7de3 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:32.866930 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10bfe2-ebbc-4977-9aae-2d4ef98a7de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:32.876644 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ceb38-4e03-4ac9-981a-b40a5fb88ba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:32.878102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ceb38-4e03-4ac9-981a-b40a5fb88ba2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:32.884546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-4e10bfe2-ebbc-4977-9aae-2d4ef98a7de3 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:32.885731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4e10bfe2-ebbc-4977-9aae-2d4ef98a7de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-2e0ceb38-4e03-4ac9-981a-b40a5fb88ba2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:32.895907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:32.896821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:32.897665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:32.897665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:32.897665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:32.897665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 229/467] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:32.897892 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ceb38-4e03-4ac9-981a-b40a5fb88ba2 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:32.979707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 239/468] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:32.979707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10bfe2-ebbc-4977-9aae-2d4ef98a7de3 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:33.042393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ceb38-4e03-4ac9-981a-b40a5fb88ba2 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:33.060019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:33.086954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f031d71-f3b5-4c1e-bc0a-245a0a04306b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:33.087185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10bfe2-ebbc-4977-9aae-2d4ef98a7de3 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:33.087594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f031d71-f3b5-4c1e-bc0a-245a0a04306b None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:33.087894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f031d71-f3b5-4c1e-bc0a-245a0a04306b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:33.088163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f031d71-f3b5-4c1e-bc0a-245a0a04306b None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:33.089080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f031d71-f3b5-4c1e-bc0a-245a0a04306b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:33.090396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:33.092571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f031d71-f3b5-4c1e-bc0a-245a0a04306b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:33.121885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 230/469] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:38:32 2022] POST /identity/v3/users => generated 246 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:38:33.123866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f031d71-f3b5-4c1e-bc0a-245a0a04306b None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:33.136229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a15ac-28bf-4577-b9ae-671cf158b1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:33.136939 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a15ac-28bf-4577-b9ae-671cf158b1df None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:33.137287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a15ac-28bf-4577-b9ae-671cf158b1df None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:33.137624 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a15ac-28bf-4577-b9ae-671cf158b1df None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:33.138366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a15ac-28bf-4577-b9ae-671cf158b1df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:33.140095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a15ac-28bf-4577-b9ae-671cf158b1df None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:33.195252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 240/470] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:38:32 2022] POST /identity/v3/users => generated 248 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:38:33.233118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78af531-1554-4947-82a5-a9651af99079 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:33.233467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78af531-1554-4947-82a5-a9651af99079 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:33.233663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78af531-1554-4947-82a5-a9651af99079 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:33.238237 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 241/471] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:38:33.241288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e6c35-8238-4ea9-a087-5481420879da None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:33.241597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e6c35-8238-4ea9-a087-5481420879da None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:33.241841 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e6c35-8238-4ea9-a087-5481420879da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:33.253619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f031d71-f3b5-4c1e-bc0a-245a0a04306b None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:33.280090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-160e6c35-8238-4ea9-a087-5481420879da None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:33.281958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-160e6c35-8238-4ea9-a087-5481420879da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:33.306167 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee5729-9be0-493d-99a5-e93bc4fcbaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:33.306796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee5729-9be0-493d-99a5-e93bc4fcbaf8 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:33.307068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee5729-9be0-493d-99a5-e93bc4fcbaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:33.307330 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee5729-9be0-493d-99a5-e93bc4fcbaf8 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:33.309113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee5729-9be0-493d-99a5-e93bc4fcbaf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:33.309444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee5729-9be0-493d-99a5-e93bc4fcbaf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:33.310871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee5729-9be0-493d-99a5-e93bc4fcbaf8 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:33.332288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 231/472] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:33.338921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee5729-9be0-493d-99a5-e93bc4fcbaf8 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:33.363009 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:33.389340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 242/473] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:33.390398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e6c35-8238-4ea9-a087-5481420879da None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:33.471569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee5729-9be0-493d-99a5-e93bc4fcbaf8 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:33.513047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e6c35-8238-4ea9-a087-5481420879da None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:33.514097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259fdec-01eb-4426-a06c-1c5d5923e38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:33.514733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259fdec-01eb-4426-a06c-1c5d5923e38a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:33.515006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259fdec-01eb-4426-a06c-1c5d5923e38a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:33.515276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259fdec-01eb-4426-a06c-1c5d5923e38a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:33.516543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:33.530211 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259fdec-01eb-4426-a06c-1c5d5923e38a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:33.532305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259fdec-01eb-4426-a06c-1c5d5923e38a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-5259fdec-01eb-4426-a06c-1c5d5923e38a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:33.549482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:33.550517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:33.551370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:33.551370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:33.551370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:33.553423 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 232/474] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:33.557795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2b7d2-220d-4449-b4fa-c400018ec02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:33.557795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2b7d2-220d-4449-b4fa-c400018ec02c None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:33.557795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2b7d2-220d-4449-b4fa-c400018ec02c None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:33.557795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2b7d2-220d-4449-b4fa-c400018ec02c None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:33.558202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2b7d2-220d-4449-b4fa-c400018ec02c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:33.558202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2b7d2-220d-4449-b4fa-c400018ec02c None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:33.559316 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259fdec-01eb-4426-a06c-1c5d5923e38a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:33.587866 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 243/475] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:38:33 2022] POST /identity/v3/users => generated 248 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:38:33.690689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:33.722079 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259fdec-01eb-4426-a06c-1c5d5923e38a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:33.760794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8becab-b168-4aa8-b4ba-15293340949d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:33.761726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8becab-b168-4aa8-b4ba-15293340949d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:33.762008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8becab-b168-4aa8-b4ba-15293340949d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:33.762109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2b7d2-220d-4449-b4fa-c400018ec02c None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:33.762336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8becab-b168-4aa8-b4ba-15293340949d None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:33.763091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8becab-b168-4aa8-b4ba-15293340949d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:33.763430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8becab-b168-4aa8-b4ba-15293340949d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:33.764848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8becab-b168-4aa8-b4ba-15293340949d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:33.779647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 233/476] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:33.783776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8becab-b168-4aa8-b4ba-15293340949d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:33.901522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2b7d2-220d-4449-b4fa-c400018ec02c None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:33.924682 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8becab-b168-4aa8-b4ba-15293340949d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:33.956899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518aef9a-84c9-4cc8-b59e-15b8a3be6083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:33.957519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aef9a-84c9-4cc8-b59e-15b8a3be6083 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:33.957790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aef9a-84c9-4cc8-b59e-15b8a3be6083 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:33.958061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aef9a-84c9-4cc8-b59e-15b8a3be6083 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:33.958736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518aef9a-84c9-4cc8-b59e-15b8a3be6083 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:33.960134 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518aef9a-84c9-4cc8-b59e-15b8a3be6083 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:33.967203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:33.977440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015b71b-dccd-4cd1-9d43-8810e8417aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:33.977985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015b71b-dccd-4cd1-9d43-8810e8417aff None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:33.978259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015b71b-dccd-4cd1-9d43-8810e8417aff None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:33.978522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015b71b-dccd-4cd1-9d43-8810e8417aff None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:33.979155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015b71b-dccd-4cd1-9d43-8810e8417aff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:33.980607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015b71b-dccd-4cd1-9d43-8810e8417aff None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:33.988482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.sql.core [None req-518aef9a-84c9-4cc8-b59e-15b8a3be6083 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 14:38:33.988482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 18 14:38:33.988482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [parameters: {'id': 'c3fd06429b284f749ab9a43969224d94', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 18 14:38:33.988482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93179) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-518aef9a-84c9-4cc8-b59e-15b8a3be6083 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application conn.query(q) Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application result.read() Aug 18 14:38:34.012076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return ref.to_dict() Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 14:38:34.013046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application next(self.gen) Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application yield resource Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application next(self.gen) Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise self.value Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application session.commit() Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 14:38:34.013955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self.session.flush() Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application compat.raise_( Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 14:38:34.016142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 14:38:34.017825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application conn.query(q) Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application result.read() Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 14:38:34.019313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [parameters: {'id': 'c3fd06429b284f749ab9a43969224d94', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:34.028823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 18 14:38:34.030882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:34.033546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 244/477] 10.222.0.36 () {60 vars in 1131 bytes} [Thu Aug 18 14:38:33 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:34.033702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 234/478] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:38:33 2022] POST /identity/v3/users => generated 249 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:38:34.033853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518aef9a-84c9-4cc8-b59e-15b8a3be6083 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:34.057547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:34.204539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518aef9a-84c9-4cc8-b59e-15b8a3be6083 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:34.241089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebb424-96b9-4ddf-987a-63728db34080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:34.241704 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebb424-96b9-4ddf-987a-63728db34080 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:34.241984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebb424-96b9-4ddf-987a-63728db34080 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:34.242255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebb424-96b9-4ddf-987a-63728db34080 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:34.253002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebb424-96b9-4ddf-987a-63728db34080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-ccebb424-96b9-4ddf-987a-63728db34080 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:34.258107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:34.260183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:34.262213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 245/479] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:38:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:34.270219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015b71b-dccd-4cd1-9d43-8810e8417aff None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:34.381468 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015b71b-dccd-4cd1-9d43-8810e8417aff None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:34.419839 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fcdd2-ab8c-41fb-81e4-70de0ce09f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:34.420463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fcdd2-ab8c-41fb-81e4-70de0ce09f5e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:34.420731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fcdd2-ab8c-41fb-81e4-70de0ce09f5e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:34.420995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fcdd2-ab8c-41fb-81e4-70de0ce09f5e None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:34.421658 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fcdd2-ab8c-41fb-81e4-70de0ce09f5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:34.421953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fcdd2-ab8c-41fb-81e4-70de0ce09f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:34.423890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fcdd2-ab8c-41fb-81e4-70de0ce09f5e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:34.433098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 235/480] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:35.552132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05670cb-4e34-4af3-905c-0e52bd078945 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:35.553166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05670cb-4e34-4af3-905c-0e52bd078945 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:35.553649 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05670cb-4e34-4af3-905c-0e52bd078945 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:35.555601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 246/481] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:35.561069 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4c14f-3179-42e5-929b-a3a5fc5617b9 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:35.562004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4c14f-3179-42e5-929b-a3a5fc5617b9 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:35.562503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4c14f-3179-42e5-929b-a3a5fc5617b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:35.587564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-96b4c14f-3179-42e5-929b-a3a5fc5617b9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:35.588808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-96b4c14f-3179-42e5-929b-a3a5fc5617b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:35.625251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24594dc5-51ae-424f-877d-873ce73de16d None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:35.625871 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24594dc5-51ae-424f-877d-873ce73de16d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:35.626361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24594dc5-51ae-424f-877d-873ce73de16d None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:35.632218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 247/482] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:35.645690 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fa02e-f1c9-4b1d-830f-a0876bda85a1 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:35.646235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fa02e-f1c9-4b1d-830f-a0876bda85a1 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:35.646704 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fa02e-f1c9-4b1d-830f-a0876bda85a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:35.647575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:35.679145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 236/483] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:35.680375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-df1fa02e-f1c9-4b1d-830f-a0876bda85a1 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:35.681578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-df1fa02e-f1c9-4b1d-830f-a0876bda85a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:35.783314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:35.788072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 248/484] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:35.981461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4c14f-3179-42e5-929b-a3a5fc5617b9 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:36.090571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4c14f-3179-42e5-929b-a3a5fc5617b9 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:36.129053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b05f9c-aee9-4089-b6cf-b418732d41ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:36.130048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b05f9c-aee9-4089-b6cf-b418732d41ce None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.130572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b05f9c-aee9-4089-b6cf-b418732d41ce None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.131045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b05f9c-aee9-4089-b6cf-b418732d41ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:36.142192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b05f9c-aee9-4089-b6cf-b418732d41ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:36.144774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fa02e-f1c9-4b1d-830f-a0876bda85a1 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-d0b05f9c-aee9-4089-b6cf-b418732d41ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:36.148811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:36.149706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:36.150534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:36.150534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:36.150534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:36.150534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:36.150534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:36.150534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:36.150534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:36.150534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:36.150534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:36.150534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:36.150534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:36.150534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:36.153574 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 237/485] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:36.160003 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b05f9c-aee9-4089-b6cf-b418732d41ce None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:36.203138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:36.251862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fa02e-f1c9-4b1d-830f-a0876bda85a1 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:36.269343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b05f9c-aee9-4089-b6cf-b418732d41ce None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:36.287260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f451f3-9af1-43f6-af01-23d26b5c93cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:36.287920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f451f3-9af1-43f6-af01-23d26b5c93cc None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.288198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f451f3-9af1-43f6-af01-23d26b5c93cc None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.288468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f451f3-9af1-43f6-af01-23d26b5c93cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:36.298942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f451f3-9af1-43f6-af01-23d26b5c93cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-20f451f3-9af1-43f6-af01-23d26b5c93cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:36.303830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:36.304707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:36.305523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 249/486] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:36.306639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36653e-fcc3-4d1d-a755-720567ea1cd4 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.306639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36653e-fcc3-4d1d-a755-720567ea1cd4 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.306874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36653e-fcc3-4d1d-a755-720567ea1cd4 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:36.310839 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a5ae0-9e84-4fb8-a211-e8ead9cd8349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:36.311703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a5ae0-9e84-4fb8-a211-e8ead9cd8349 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.312187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a5ae0-9e84-4fb8-a211-e8ead9cd8349 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.313220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a5ae0-9e84-4fb8-a211-e8ead9cd8349 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:36.313675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 250/487] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:36.314302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a5ae0-9e84-4fb8-a211-e8ead9cd8349 None admin] RBAC: Adding query filter params (name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:36.314418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36653e-fcc3-4d1d-a755-720567ea1cd4 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:36.316378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a5ae0-9e84-4fb8-a211-e8ead9cd8349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:36.318204 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a5ae0-9e84-4fb8-a211-e8ead9cd8349 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:36.327097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 238/488] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:36.333327 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a516f-901b-494b-a6b8-fd8b55bab526 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.333846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a516f-901b-494b-a6b8-fd8b55bab526 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.334338 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a516f-901b-494b-a6b8-fd8b55bab526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:36.356341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-e50a516f-901b-494b-a6b8-fd8b55bab526 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:36.357962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e50a516f-901b-494b-a6b8-fd8b55bab526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:36.379147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:36.422810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36653e-fcc3-4d1d-a755-720567ea1cd4 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:36.436768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 239/489] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:36.438058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a516f-901b-494b-a6b8-fd8b55bab526 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:36.458583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf64a9-0353-4bc2-9477-13c80abe2727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:36.459338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf64a9-0353-4bc2-9477-13c80abe2727 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.459534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf64a9-0353-4bc2-9477-13c80abe2727 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.459828 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf64a9-0353-4bc2-9477-13c80abe2727 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:36.460496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf64a9-0353-4bc2-9477-13c80abe2727 None admin] RBAC: Adding query filter params (name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:36.460798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf64a9-0353-4bc2-9477-13c80abe2727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:36.462293 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf64a9-0353-4bc2-9477-13c80abe2727 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:36.471560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 251/490] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:36.473511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:36.477191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf64a9-0353-4bc2-9477-13c80abe2727 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:36.550239 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a516f-901b-494b-a6b8-fd8b55bab526 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:36.577570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf64a9-0353-4bc2-9477-13c80abe2727 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:36.587505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363f94b-7bee-4533-b6b7-1cae800adc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:36.588726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363f94b-7bee-4533-b6b7-1cae800adc0f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.589228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363f94b-7bee-4533-b6b7-1cae800adc0f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.589720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363f94b-7bee-4533-b6b7-1cae800adc0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:36.600681 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363f94b-7bee-4533-b6b7-1cae800adc0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:36.602398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363f94b-7bee-4533-b6b7-1cae800adc0f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:36.608067 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:36.616358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a14e3-b002-427c-93ee-ed8de4af759a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:36.616925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a14e3-b002-427c-93ee-ed8de4af759a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.617266 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a14e3-b002-427c-93ee-ed8de4af759a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.617551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a14e3-b002-427c-93ee-ed8de4af759a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-b363f94b-7bee-4533-b6b7-1cae800adc0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:36.621730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:36.622694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:36.623841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:36.623841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:36.623841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:36.630447 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a14e3-b002-427c-93ee-ed8de4af759a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:36.630712 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 240/491] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:36.630823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a14e3-b002-427c-93ee-ed8de4af759a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:36.632487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10baf336-f5fd-48ac-b7f2-978a418b3acd None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.632971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10baf336-f5fd-48ac-b7f2-978a418b3acd None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.633462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10baf336-f5fd-48ac-b7f2-978a418b3acd None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:36.635168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 241/492] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:36.636137 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10baf336-f5fd-48ac-b7f2-978a418b3acd None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-645a14e3-b002-427c-93ee-ed8de4af759a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:36.647397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:36.648400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:36.649252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:36.649252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:36.649252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:36.649252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 252/493] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:36.650109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3743a5-05bc-4ae5-bf89-603f8076caf2 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.650109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3743a5-05bc-4ae5-bf89-603f8076caf2 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.650264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3743a5-05bc-4ae5-bf89-603f8076caf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:36.674093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-da3743a5-05bc-4ae5-bf89-603f8076caf2 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:36.675310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-da3743a5-05bc-4ae5-bf89-603f8076caf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:36.732766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:36.745355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 253/494] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:36.746454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3743a5-05bc-4ae5-bf89-603f8076caf2 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:36.763956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10baf336-f5fd-48ac-b7f2-978a418b3acd None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:36.801012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07a79d-28b3-4667-847d-2076880707eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:36.801617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07a79d-28b3-4667-847d-2076880707eb None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.801896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07a79d-28b3-4667-847d-2076880707eb None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.802165 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07a79d-28b3-4667-847d-2076880707eb None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:36.802797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07a79d-28b3-4667-847d-2076880707eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:36.803106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07a79d-28b3-4667-847d-2076880707eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:36.804622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07a79d-28b3-4667-847d-2076880707eb None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:36.817990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 242/495] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:36.818923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07a79d-28b3-4667-847d-2076880707eb None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:36.865588 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3743a5-05bc-4ae5-bf89-603f8076caf2 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:36.869219 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:36.923907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80085de-8b6f-476a-acb3-99371fc6bccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:36.925278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80085de-8b6f-476a-acb3-99371fc6bccf None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.925713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80085de-8b6f-476a-acb3-99371fc6bccf None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.926149 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80085de-8b6f-476a-acb3-99371fc6bccf None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:36.926957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80085de-8b6f-476a-acb3-99371fc6bccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:36.927416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80085de-8b6f-476a-acb3-99371fc6bccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:36.929799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80085de-8b6f-476a-acb3-99371fc6bccf None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:36.943573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07a79d-28b3-4667-847d-2076880707eb None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:36.946070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 254/496] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:36.947178 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80085de-8b6f-476a-acb3-99371fc6bccf None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:36.996353 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb5ace-5677-4c35-aea3-ec7f3eed16ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:36.997970 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb5ace-5677-4c35-aea3-ec7f3eed16ac None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:36.998611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb5ace-5677-4c35-aea3-ec7f3eed16ac None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:36.998953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb5ace-5677-4c35-aea3-ec7f3eed16ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.011374 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb5ace-5677-4c35-aea3-ec7f3eed16ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-c3cb5ace-5677-4c35-aea3-ec7f3eed16ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:37.016081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:37.016999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:37.017783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 243/497] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:37.018448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb5ace-5677-4c35-aea3-ec7f3eed16ac None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:37.021541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:37.069166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80085de-8b6f-476a-acb3-99371fc6bccf None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:37.101246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f0347-ce6c-4ebc-8ec2-94e2a99a9968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:37.101870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f0347-ce6c-4ebc-8ec2-94e2a99a9968 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:37.102145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f0347-ce6c-4ebc-8ec2-94e2a99a9968 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:37.102415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f0347-ce6c-4ebc-8ec2-94e2a99a9968 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.110131 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f0347-ce6c-4ebc-8ec2-94e2a99a9968 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:37.111575 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f0347-ce6c-4ebc-8ec2-94e2a99a9968 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:37.119448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb5ace-5677-4c35-aea3-ec7f3eed16ac None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-f81f0347-ce6c-4ebc-8ec2-94e2a99a9968 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:37.123891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:37.124839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:37.125779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:37.125779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:37.125779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:37.125779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:37.125779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:37.125779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:37.125779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:37.125779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:37.125779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:38:37.125779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:37.125779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 255/498] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Aug 18 14:38:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:37.126348 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97e479-9d3d-436c-856d-02183f26f4e9 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:37.126348 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97e479-9d3d-436c-856d-02183f26f4e9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:37.126780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97e479-9d3d-436c-856d-02183f26f4e9 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.127985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 256/499] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:37.128502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97e479-9d3d-436c-856d-02183f26f4e9 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:37.152679 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:37.158896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234ba85-01d4-4f63-9072-4c3e29865680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:37.159521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234ba85-01d4-4f63-9072-4c3e29865680 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:37.159837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234ba85-01d4-4f63-9072-4c3e29865680 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:37.160114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234ba85-01d4-4f63-9072-4c3e29865680 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.167053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234ba85-01d4-4f63-9072-4c3e29865680 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:37.168521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234ba85-01d4-4f63-9072-4c3e29865680 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-d234ba85-01d4-4f63-9072-4c3e29865680 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:37.181178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:37.182105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:37.188392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:37.188392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:37.188392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:37.188392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:37.188392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:37.188392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:37.188392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:37.188392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:37.188392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 14:38:37.188392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:37.188392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 244/500] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:37.188392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234ba85-01d4-4f63-9072-4c3e29865680 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:37.230910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97e479-9d3d-436c-856d-02183f26f4e9 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:37.246350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:37.264351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0917f-ffd2-47b4-b680-f63640a0e2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:37.264661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0917f-ffd2-47b4-b680-f63640a0e2d8 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:37.264934 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0917f-ffd2-47b4-b680-f63640a0e2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:37.265202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0917f-ffd2-47b4-b680-f63640a0e2d8 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.265882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0917f-ffd2-47b4-b680-f63640a0e2d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:37.266177 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0917f-ffd2-47b4-b680-f63640a0e2d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:37.267543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0917f-ffd2-47b4-b680-f63640a0e2d8 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:37.275891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 257/501] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:37.276717 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0917f-ffd2-47b4-b680-f63640a0e2d8 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:37.287702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234ba85-01d4-4f63-9072-4c3e29865680 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:37.321785 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b26bc-05ac-48fb-8fed-11d8e27de583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:37.322416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b26bc-05ac-48fb-8fed-11d8e27de583 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:37.322697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b26bc-05ac-48fb-8fed-11d8e27de583 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:37.322969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b26bc-05ac-48fb-8fed-11d8e27de583 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.333177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b26bc-05ac-48fb-8fed-11d8e27de583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-f98b26bc-05ac-48fb-8fed-11d8e27de583 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:37.339724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:37.340627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:37.341428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 245/502] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:37.342106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d8144-ec65-4794-af59-d169c90465d4 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:37.342172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d8144-ec65-4794-af59-d169c90465d4 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:37.342471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d8144-ec65-4794-af59-d169c90465d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.359362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-b87d8144-ec65-4794-af59-d169c90465d4 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:37.361206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b87d8144-ec65-4794-af59-d169c90465d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:37.365031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:37.380676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0917f-ffd2-47b4-b680-f63640a0e2d8 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:37.413099 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d922bee-8224-49c8-895d-9bcdd788e55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:37.413695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d922bee-8224-49c8-895d-9bcdd788e55f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:37.413985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d922bee-8224-49c8-895d-9bcdd788e55f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:37.414254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d922bee-8224-49c8-895d-9bcdd788e55f None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.414965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d922bee-8224-49c8-895d-9bcdd788e55f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:37.415263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d922bee-8224-49c8-895d-9bcdd788e55f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:37.416707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d922bee-8224-49c8-895d-9bcdd788e55f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:37.429391 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 246/503] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:37.430230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d8144-ec65-4794-af59-d169c90465d4 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:37.453259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 258/504] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Aug 18 14:38:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:37.454011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d922bee-8224-49c8-895d-9bcdd788e55f None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:37.492408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:37.544264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d8144-ec65-4794-af59-d169c90465d4 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:37.554097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d922bee-8224-49c8-895d-9bcdd788e55f None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:37.580838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce83a3-9f5c-4490-a033-71ef1ecfe06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:37.581841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce83a3-9f5c-4490-a033-71ef1ecfe06f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:37.582355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce83a3-9f5c-4490-a033-71ef1ecfe06f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:37.582875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce83a3-9f5c-4490-a033-71ef1ecfe06f None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.583799 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce83a3-9f5c-4490-a033-71ef1ecfe06f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:37.584177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce83a3-9f5c-4490-a033-71ef1ecfe06f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:37.585620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce83a3-9f5c-4490-a033-71ef1ecfe06f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:37.587225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ea5b8-20df-44e1-9c2b-5309d795ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:37.587854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ea5b8-20df-44e1-9c2b-5309d795ac4d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:37.588136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ea5b8-20df-44e1-9c2b-5309d795ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:37.588405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ea5b8-20df-44e1-9c2b-5309d795ac4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.598539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ea5b8-20df-44e1-9c2b-5309d795ac4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:37.600078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ea5b8-20df-44e1-9c2b-5309d795ac4d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:37.602497 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-251ea5b8-20df-44e1-9c2b-5309d795ac4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:37.615946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:37.616891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:37.622572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:37.622572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:37.622572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:37.622572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 259/505] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:37.622572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ea5b8-20df-44e1-9c2b-5309d795ac4d None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:37.632100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 247/506] 10.222.0.36 () {56 vars in 1137 bytes} [Thu Aug 18 14:38:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:37.632100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce83a3-9f5c-4490-a033-71ef1ecfe06f None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:37.731247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce83a3-9f5c-4490-a033-71ef1ecfe06f None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:37.734239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ea5b8-20df-44e1-9c2b-5309d795ac4d None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:37.741924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:37.765799 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89952db0-e09e-49de-b31b-125e2aec81cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:37.766397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89952db0-e09e-49de-b31b-125e2aec81cb None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:37.766609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7ea26-927d-446c-8479-c453d3579da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:37.766717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89952db0-e09e-49de-b31b-125e2aec81cb None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:37.766935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89952db0-e09e-49de-b31b-125e2aec81cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.767204 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7ea26-927d-446c-8479-c453d3579da3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:37.767481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7ea26-927d-446c-8479-c453d3579da3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:37.767854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7ea26-927d-446c-8479-c453d3579da3 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.768443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7ea26-927d-446c-8479-c453d3579da3 None admin] RBAC: Adding query filter params (name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:37.768748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7ea26-927d-446c-8479-c453d3579da3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:37.770124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7ea26-927d-446c-8479-c453d3579da3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:37.777748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 260/507] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:37.778526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7ea26-927d-446c-8479-c453d3579da3 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:37.834658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:37.838108 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89952db0-e09e-49de-b31b-125e2aec81cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:37.839594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89952db0-e09e-49de-b31b-125e2aec81cb None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:37.876696 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7ea26-927d-446c-8479-c453d3579da3 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:37.912133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f23e3-a503-4cc6-bb03-436cd6915349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:37.912737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f23e3-a503-4cc6-bb03-436cd6915349 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:37.913017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f23e3-a503-4cc6-bb03-436cd6915349 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:37.913289 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f23e3-a503-4cc6-bb03-436cd6915349 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:37.913956 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f23e3-a503-4cc6-bb03-436cd6915349 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:37.914256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f23e3-a503-4cc6-bb03-436cd6915349 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:37.915623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f23e3-a503-4cc6-bb03-436cd6915349 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:37.927192 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 261/508] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:37.928040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f23e3-a503-4cc6-bb03-436cd6915349 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:37.946157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 248/509] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:37.946157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89952db0-e09e-49de-b31b-125e2aec81cb None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:37.954947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:38.025813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f23e3-a503-4cc6-bb03-436cd6915349 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:38.042363 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:38.042514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89952db0-e09e-49de-b31b-125e2aec81cb None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:38.059494 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bb6ab-0727-4dc1-8b65-d448af514d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:38.060201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bb6ab-0727-4dc1-8b65-d448af514d92 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:38.060484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bb6ab-0727-4dc1-8b65-d448af514d92 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:38.060807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bb6ab-0727-4dc1-8b65-d448af514d92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:38.076074 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429fd5a-d343-49c9-bbfe-cbd8194f6f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:38.076689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429fd5a-d343-49c9-bbfe-cbd8194f6f91 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:38.076965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429fd5a-d343-49c9-bbfe-cbd8194f6f91 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:38.077232 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429fd5a-d343-49c9-bbfe-cbd8194f6f91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:38.087529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429fd5a-d343-49c9-bbfe-cbd8194f6f91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:38.089017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429fd5a-d343-49c9-bbfe-cbd8194f6f91 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-e429fd5a-d343-49c9-bbfe-cbd8194f6f91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:38.103613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:38.104692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:38.108853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:38.108853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:38.108853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:38.108853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 249/510] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:38.108853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429fd5a-d343-49c9-bbfe-cbd8194f6f91 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:38.130984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bb6ab-0727-4dc1-8b65-d448af514d92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:38.132485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bb6ab-0727-4dc1-8b65-d448af514d92 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:38.166311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:38.204400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429fd5a-d343-49c9-bbfe-cbd8194f6f91 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:38.228652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 262/511] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:38.229481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bb6ab-0727-4dc1-8b65-d448af514d92 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:38.241583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20330e74-5e6e-489f-af0f-386092465fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:38.242285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20330e74-5e6e-489f-af0f-386092465fcd None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:38.242635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20330e74-5e6e-489f-af0f-386092465fcd None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:38.242981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20330e74-5e6e-489f-af0f-386092465fcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:38.247708 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:38.254247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20330e74-5e6e-489f-af0f-386092465fcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:38.255760 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20330e74-5e6e-489f-af0f-386092465fcd None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-20330e74-5e6e-489f-af0f-386092465fcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:38.269476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:38.270558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:38.271409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:38.271409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:38.271409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:38.271409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 250/512] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:38.271409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20330e74-5e6e-489f-af0f-386092465fcd None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:38.331052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bb6ab-0727-4dc1-8b65-d448af514d92 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:38.365218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fc95b-4fc0-4966-84e7-d34c61ea7035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:38.365989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fc95b-4fc0-4966-84e7-d34c61ea7035 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:38.366383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fc95b-4fc0-4966-84e7-d34c61ea7035 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:38.366734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fc95b-4fc0-4966-84e7-d34c61ea7035 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:38.369898 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20330e74-5e6e-489f-af0f-386092465fcd None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:38.373807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fc95b-4fc0-4966-84e7-d34c61ea7035 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:38.375778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fc95b-4fc0-4966-84e7-d34c61ea7035 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:38.383795 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-2f0fc95b-4fc0-4966-84e7-d34c61ea7035 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:38.387315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:38.388457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:38.389523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:38.389523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:38.389523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:38.389523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:38.389523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:38.389523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:38.389523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:38.389523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:38.389523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 14:38:38.389523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:38.389523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 263/513] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:38.389523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fc95b-4fc0-4966-84e7-d34c61ea7035 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:38.409519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59351d0b-c225-481c-acb7-514f3d1d3a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:38.410141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59351d0b-c225-481c-acb7-514f3d1d3a71 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:38.410418 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59351d0b-c225-481c-acb7-514f3d1d3a71 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:38.410694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59351d0b-c225-481c-acb7-514f3d1d3a71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:38.420993 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59351d0b-c225-481c-acb7-514f3d1d3a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:38.422485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59351d0b-c225-481c-acb7-514f3d1d3a71 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-59351d0b-c225-481c-acb7-514f3d1d3a71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:38.436245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:38.437310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:38.438854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:38.438854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:38.438854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:38.438854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 251/514] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:38.438854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59351d0b-c225-481c-acb7-514f3d1d3a71 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:38.473728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:38.494538 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fc95b-4fc0-4966-84e7-d34c61ea7035 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:38.527724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c5dcc-cc1f-422a-a1c9-fbc22827e823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:38.528388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c5dcc-cc1f-422a-a1c9-fbc22827e823 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:38.528667 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c5dcc-cc1f-422a-a1c9-fbc22827e823 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:38.528941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c5dcc-cc1f-422a-a1c9-fbc22827e823 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:38.529574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c5dcc-cc1f-422a-a1c9-fbc22827e823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:38.529874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c5dcc-cc1f-422a-a1c9-fbc22827e823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:38.531238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c5dcc-cc1f-422a-a1c9-fbc22827e823 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:38.536613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59351d0b-c225-481c-acb7-514f3d1d3a71 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:38.542575 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 264/515] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:38.543360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c5dcc-cc1f-422a-a1c9-fbc22827e823 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:38.573796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6753f5b-cbb3-413a-8b3c-5c7c22c6fce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:38.574497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6753f5b-cbb3-413a-8b3c-5c7c22c6fce2 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:38.574854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6753f5b-cbb3-413a-8b3c-5c7c22c6fce2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:38.575190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6753f5b-cbb3-413a-8b3c-5c7c22c6fce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:38.585986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6753f5b-cbb3-413a-8b3c-5c7c22c6fce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:38.587371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6753f5b-cbb3-413a-8b3c-5c7c22c6fce2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-c6753f5b-cbb3-413a-8b3c-5c7c22c6fce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:38.601122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:38.602007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:38.602859 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:38.602859 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:38.602859 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:38.602859 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 252/516] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:38.603070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6753f5b-cbb3-413a-8b3c-5c7c22c6fce2 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:38.621756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:38.643633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c5dcc-cc1f-422a-a1c9-fbc22827e823 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:38.676862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df062be9-9026-4a28-bc2b-43e4e216b246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:38.677463 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df062be9-9026-4a28-bc2b-43e4e216b246 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:38.677735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df062be9-9026-4a28-bc2b-43e4e216b246 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:38.678005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df062be9-9026-4a28-bc2b-43e4e216b246 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:38.678633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df062be9-9026-4a28-bc2b-43e4e216b246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:38.678933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df062be9-9026-4a28-bc2b-43e4e216b246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:38.680342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df062be9-9026-4a28-bc2b-43e4e216b246 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:38.686729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:38.692153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 265/517] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:38.692896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df062be9-9026-4a28-bc2b-43e4e216b246 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:38.703141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6753f5b-cbb3-413a-8b3c-5c7c22c6fce2 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:38.737837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b1077-81ca-4046-af56-06000fd6088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:38.738436 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b1077-81ca-4046-af56-06000fd6088e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:38.738713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b1077-81ca-4046-af56-06000fd6088e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:38.738984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b1077-81ca-4046-af56-06000fd6088e None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:38.739721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b1077-81ca-4046-af56-06000fd6088e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:38.740032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b1077-81ca-4046-af56-06000fd6088e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:38.741477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b1077-81ca-4046-af56-06000fd6088e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:38.758995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 253/518] 10.222.0.36 () {56 vars in 1137 bytes} [Thu Aug 18 14:38:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:38.759789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b1077-81ca-4046-af56-06000fd6088e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:38.792749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df062be9-9026-4a28-bc2b-43e4e216b246 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:38.827014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e43c8-9fba-49d6-a544-2fa0959b8dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:38.827616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e43c8-9fba-49d6-a544-2fa0959b8dfe None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:38.827921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e43c8-9fba-49d6-a544-2fa0959b8dfe None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:38.828194 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e43c8-9fba-49d6-a544-2fa0959b8dfe None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:38.828825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e43c8-9fba-49d6-a544-2fa0959b8dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:38.829125 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e43c8-9fba-49d6-a544-2fa0959b8dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:38.830502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e43c8-9fba-49d6-a544-2fa0959b8dfe None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:38.833550 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:38.842061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 266/519] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:38.842784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e43c8-9fba-49d6-a544-2fa0959b8dfe None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:38.857399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b1077-81ca-4046-af56-06000fd6088e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:38.891335 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca520932-c59a-4eca-9494-0c1ab49ef85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:38.891970 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca520932-c59a-4eca-9494-0c1ab49ef85f None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:38.894774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca520932-c59a-4eca-9494-0c1ab49ef85f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:38.894774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca520932-c59a-4eca-9494-0c1ab49ef85f None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:38.894774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca520932-c59a-4eca-9494-0c1ab49ef85f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:38.894774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca520932-c59a-4eca-9494-0c1ab49ef85f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:38.901093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:38.914779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 254/520] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:38:38 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:38:38.915584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca520932-c59a-4eca-9494-0c1ab49ef85f None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:38.964530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e43c8-9fba-49d6-a544-2fa0959b8dfe None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:39.006728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e762ae-0c91-40d4-b04f-55652e8b48a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:39.007668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e762ae-0c91-40d4-b04f-55652e8b48a6 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:39.008193 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e762ae-0c91-40d4-b04f-55652e8b48a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:39.008466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e762ae-0c91-40d4-b04f-55652e8b48a6 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:39.009121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e762ae-0c91-40d4-b04f-55652e8b48a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:39.009423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e762ae-0c91-40d4-b04f-55652e8b48a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:39.010833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e762ae-0c91-40d4-b04f-55652e8b48a6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:39.023715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 267/521] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:39.024519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e762ae-0c91-40d4-b04f-55652e8b48a6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:39.039904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca520932-c59a-4eca-9494-0c1ab49ef85f None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:39.071984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:39.081706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a58b09-6eb7-476d-9fe9-86ca612f90a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:39.082549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a58b09-6eb7-476d-9fe9-86ca612f90a5 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:39.082974 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a58b09-6eb7-476d-9fe9-86ca612f90a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:39.083384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a58b09-6eb7-476d-9fe9-86ca612f90a5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:39.094155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a58b09-6eb7-476d-9fe9-86ca612f90a5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:39.094778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a58b09-6eb7-476d-9fe9-86ca612f90a5 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-a1a58b09-6eb7-476d-9fe9-86ca612f90a5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:39.107401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:39.108385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:39.109295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:39.109295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:39.109295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:39.109295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:39.109295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:39.109295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:39.109295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:39.109295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:39.109295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 14:38:39.109295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:39.109295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 255/522] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:39.110336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a58b09-6eb7-476d-9fe9-86ca612f90a5 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:39.146478 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e762ae-0c91-40d4-b04f-55652e8b48a6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:39.164102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:39.181364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bec39-9c3a-43aa-8e8e-baf135c7a6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:39.181974 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bec39-9c3a-43aa-8e8e-baf135c7a6f1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:39.182259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bec39-9c3a-43aa-8e8e-baf135c7a6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:39.182535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bec39-9c3a-43aa-8e8e-baf135c7a6f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:39.214727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a58b09-6eb7-476d-9fe9-86ca612f90a5 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:39.248608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7ec32-53f6-47b3-86e7-9435b54c594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:39.249239 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7ec32-53f6-47b3-86e7-9435b54c594c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:39.249515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7ec32-53f6-47b3-86e7-9435b54c594c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:39.249783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7ec32-53f6-47b3-86e7-9435b54c594c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:39.253485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866bec39-9c3a-43aa-8e8e-baf135c7a6f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:39.255022 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866bec39-9c3a-43aa-8e8e-baf135c7a6f1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:39.257623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7ec32-53f6-47b3-86e7-9435b54c594c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:39.259050 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7ec32-53f6-47b3-86e7-9435b54c594c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-d2e7ec32-53f6-47b3-86e7-9435b54c594c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:39.271151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:39.272265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:39.273246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:39.273246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:39.273246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:39.273246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:39.273246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:39.273246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:39.273246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:39.273246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:39.273246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 256/523] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:39.273246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7ec32-53f6-47b3-86e7-9435b54c594c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:39.326558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:39.372634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 268/524] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:39.373306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bec39-9c3a-43aa-8e8e-baf135c7a6f1 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:39.378518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7ec32-53f6-47b3-86e7-9435b54c594c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:39.394775 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:39.414076 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa41c24-9c6c-4f71-9062-fee637824bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:39.415373 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa41c24-9c6c-4f71-9062-fee637824bab None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:39.415940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa41c24-9c6c-4f71-9062-fee637824bab None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:39.416823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa41c24-9c6c-4f71-9062-fee637824bab None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:39.426188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa41c24-9c6c-4f71-9062-fee637824bab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:39.428086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa41c24-9c6c-4f71-9062-fee637824bab None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-1aa41c24-9c6c-4f71-9062-fee637824bab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:39.440341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:39.441371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:39.442368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:39.442368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:39.442368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:39.442368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:39.442368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:39.442368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:39.442368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:39.442368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:39.444355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 257/525] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:39.445416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa41c24-9c6c-4f71-9062-fee637824bab None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:39.481229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bec39-9c3a-43aa-8e8e-baf135c7a6f1 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:39.516281 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165da10-8c53-4a5e-afb6-6bbd76128bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:39.516887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165da10-8c53-4a5e-afb6-6bbd76128bc6 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:39.517167 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165da10-8c53-4a5e-afb6-6bbd76128bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:39.517437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165da10-8c53-4a5e-afb6-6bbd76128bc6 None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:39.518159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165da10-8c53-4a5e-afb6-6bbd76128bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:39.518462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165da10-8c53-4a5e-afb6-6bbd76128bc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:39.519914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165da10-8c53-4a5e-afb6-6bbd76128bc6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:39.536449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 269/526] 10.222.0.36 () {56 vars in 1139 bytes} [Thu Aug 18 14:38:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:39.537237 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165da10-8c53-4a5e-afb6-6bbd76128bc6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:39.555407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa41c24-9c6c-4f71-9062-fee637824bab None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:39.571815 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:39.595567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6064fe-a334-408a-a1a0-617d21954aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:39.596226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6064fe-a334-408a-a1a0-617d21954aab None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:39.596508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6064fe-a334-408a-a1a0-617d21954aab None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:39.597019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6064fe-a334-408a-a1a0-617d21954aab None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:39.597472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6064fe-a334-408a-a1a0-617d21954aab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:39.597781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6064fe-a334-408a-a1a0-617d21954aab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:39.599246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6064fe-a334-408a-a1a0-617d21954aab None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:39.617893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 258/527] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:39.618973 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:39.621322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6064fe-a334-408a-a1a0-617d21954aab None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:39.645094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165da10-8c53-4a5e-afb6-6bbd76128bc6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:39.680327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da346d-0572-4571-a1e5-46ce6052b171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:39.680971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da346d-0572-4571-a1e5-46ce6052b171 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:39.681326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da346d-0572-4571-a1e5-46ce6052b171 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:39.681594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da346d-0572-4571-a1e5-46ce6052b171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:39.692017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da346d-0572-4571-a1e5-46ce6052b171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:39.693591 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da346d-0572-4571-a1e5-46ce6052b171 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-d2da346d-0572-4571-a1e5-46ce6052b171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:39.708937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:39.709859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:39.710802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:39.710802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:39.710802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:39.710802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 270/528] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:39.711054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da346d-0572-4571-a1e5-46ce6052b171 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:39.728558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6064fe-a334-408a-a1a0-617d21954aab None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:39.763773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ad75b-0795-449c-baef-de99e052c24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:39.764365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ad75b-0795-449c-baef-de99e052c24e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:39.764638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ad75b-0795-449c-baef-de99e052c24e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:39.764911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ad75b-0795-449c-baef-de99e052c24e None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:39.765587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ad75b-0795-449c-baef-de99e052c24e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:39.766415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ad75b-0795-449c-baef-de99e052c24e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:39.767861 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ad75b-0795-449c-baef-de99e052c24e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:39.778933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 259/529] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:39.783900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ad75b-0795-449c-baef-de99e052c24e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:39.812821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:39.818448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da346d-0572-4571-a1e5-46ce6052b171 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:39.846049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:39.854976 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765560d-5cf3-4a98-88c6-218c141cb37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:39.855634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765560d-5cf3-4a98-88c6-218c141cb37c None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:39.855943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765560d-5cf3-4a98-88c6-218c141cb37c None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:39.856210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765560d-5cf3-4a98-88c6-218c141cb37c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:39.889499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ad75b-0795-449c-baef-de99e052c24e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:39.924799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765560d-5cf3-4a98-88c6-218c141cb37c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:39.926277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765560d-5cf3-4a98-88c6-218c141cb37c None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:39.939797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5d83d-5e9d-425d-b037-31b3f71955a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:39.940754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5d83d-5e9d-425d-b037-31b3f71955a8 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:39.941535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5d83d-5e9d-425d-b037-31b3f71955a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:39.942229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5d83d-5e9d-425d-b037-31b3f71955a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:39.943132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5d83d-5e9d-425d-b037-31b3f71955a8 None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=3f3efa9f3eb24864943b68d241435817) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:39.943743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5d83d-5e9d-425d-b037-31b3f71955a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:39.945656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5d83d-5e9d-425d-b037-31b3f71955a8 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:39.959710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 260/530] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:39 2022] GET /identity/v3/role_assignments?user.id=3f3efa9f3eb24864943b68d241435817&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:39.959710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5d83d-5e9d-425d-b037-31b3f71955a8 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:40.071728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 271/531] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:40.075190 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765560d-5cf3-4a98-88c6-218c141cb37c None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:40.077130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:40.087475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5d83d-5e9d-425d-b037-31b3f71955a8 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:40.107823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:40.144627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45058ec-8cf4-416b-b629-3f23cf8e2c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:40.145379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45058ec-8cf4-416b-b629-3f23cf8e2c0a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:40.145788 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45058ec-8cf4-416b-b629-3f23cf8e2c0a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:40.146073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45058ec-8cf4-416b-b629-3f23cf8e2c0a None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:40.146782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45058ec-8cf4-416b-b629-3f23cf8e2c0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:40.147034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45058ec-8cf4-416b-b629-3f23cf8e2c0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:40.149162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45058ec-8cf4-416b-b629-3f23cf8e2c0a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:40.166267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 261/532] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:40.167114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45058ec-8cf4-416b-b629-3f23cf8e2c0a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:40.192662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765560d-5cf3-4a98-88c6-218c141cb37c None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:40.230462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955726a-07b8-4f48-b7fa-3d5a4d43581a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:40.231425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955726a-07b8-4f48-b7fa-3d5a4d43581a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:40.231939 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955726a-07b8-4f48-b7fa-3d5a4d43581a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:40.232415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955726a-07b8-4f48-b7fa-3d5a4d43581a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:40.233340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955726a-07b8-4f48-b7fa-3d5a4d43581a None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=5f2b6dc3406f46e68c74677f5fc7e3b1) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:40.233892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955726a-07b8-4f48-b7fa-3d5a4d43581a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:40.236101 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955726a-07b8-4f48-b7fa-3d5a4d43581a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:40.247972 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 272/533] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:40 2022] GET /identity/v3/role_assignments?user.id=5f2b6dc3406f46e68c74677f5fc7e3b1&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:40.249672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955726a-07b8-4f48-b7fa-3d5a4d43581a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:40.299994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45058ec-8cf4-416b-b629-3f23cf8e2c0a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:40.348203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203df34-a7e2-45f7-a95b-895fe7ada1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:40.348856 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203df34-a7e2-45f7-a95b-895fe7ada1e6 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:40.349179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203df34-a7e2-45f7-a95b-895fe7ada1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:40.349465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203df34-a7e2-45f7-a95b-895fe7ada1e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:40.363258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203df34-a7e2-45f7-a95b-895fe7ada1e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:40.365127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203df34-a7e2-45f7-a95b-895fe7ada1e6 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-8203df34-a7e2-45f7-a95b-895fe7ada1e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:40.383873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:40.384847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:40.385704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:40.385704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:40.385704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:40.385704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 262/534] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:40.393482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203df34-a7e2-45f7-a95b-895fe7ada1e6 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:40.415735 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:40.456485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:40.471193 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955726a-07b8-4f48-b7fa-3d5a4d43581a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:40.518517 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729604fa-0033-4ea4-904e-7442471160f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:40.519412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729604fa-0033-4ea4-904e-7442471160f7 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:40.520330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729604fa-0033-4ea4-904e-7442471160f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:40.520757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729604fa-0033-4ea4-904e-7442471160f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:40.530653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729604fa-0033-4ea4-904e-7442471160f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:40.532416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729604fa-0033-4ea4-904e-7442471160f7 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:40.545366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203df34-a7e2-45f7-a95b-895fe7ada1e6 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-729604fa-0033-4ea4-904e-7442471160f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:40.549657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:40.550477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:40.551326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:40.551326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:40.551326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:40.551326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:40.551326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:40.551326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:40.551326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:40.551326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:40.553047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 273/535] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:40.560636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729604fa-0033-4ea4-904e-7442471160f7 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:40.599705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc76f33-a5cc-455a-8744-86eb7d48796c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:40.599705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc76f33-a5cc-455a-8744-86eb7d48796c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:40.599705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc76f33-a5cc-455a-8744-86eb7d48796c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:40.599705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc76f33-a5cc-455a-8744-86eb7d48796c None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:40.600122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc76f33-a5cc-455a-8744-86eb7d48796c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:40.600122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc76f33-a5cc-455a-8744-86eb7d48796c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:40.600122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc76f33-a5cc-455a-8744-86eb7d48796c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:40.608486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 263/536] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:40.615330 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc76f33-a5cc-455a-8744-86eb7d48796c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:40.732151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:40.758590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729604fa-0033-4ea4-904e-7442471160f7 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:40.773829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc76f33-a5cc-455a-8744-86eb7d48796c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:40.812782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd7f5b-5311-4af5-8683-b950bac245f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:40.813022 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae2f1d-bc6d-4a2b-a026-9b04512a4ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:40.813755 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae2f1d-bc6d-4a2b-a026-9b04512a4ff7 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:40.814010 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd7f5b-5311-4af5-8683-b950bac245f9 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:40.814225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae2f1d-bc6d-4a2b-a026-9b04512a4ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:40.814602 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae2f1d-bc6d-4a2b-a026-9b04512a4ff7 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:40.814723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd7f5b-5311-4af5-8683-b950bac245f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:40.815121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd7f5b-5311-4af5-8683-b950bac245f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:40.815564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae2f1d-bc6d-4a2b-a026-9b04512a4ff7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:40.815999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae2f1d-bc6d-4a2b-a026-9b04512a4ff7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:40.817547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae2f1d-bc6d-4a2b-a026-9b04512a4ff7 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:40.823186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd7f5b-5311-4af5-8683-b950bac245f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:40.824879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd7f5b-5311-4af5-8683-b950bac245f9 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:40.836620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 274/537] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-c4dd7f5b-5311-4af5-8683-b950bac245f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:40.837162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:40.838206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:40.839332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:40.839332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:40.839332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:40.839332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:40.839332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:40.839332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:40.839332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:40.839332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:40.841291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 264/538] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:40.846419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae2f1d-bc6d-4a2b-a026-9b04512a4ff7 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:40.847517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd7f5b-5311-4af5-8683-b950bac245f9 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:40.871575 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:40.969634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd7f5b-5311-4af5-8683-b950bac245f9 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:41.003121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:41.015452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030d0db-dab0-41e3-a4ae-957af1c2c152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:41.017145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030d0db-dab0-41e3-a4ae-957af1c2c152 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:41.017576 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030d0db-dab0-41e3-a4ae-957af1c2c152 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:41.018128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030d0db-dab0-41e3-a4ae-957af1c2c152 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:41.019301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030d0db-dab0-41e3-a4ae-957af1c2c152 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:41.019912 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030d0db-dab0-41e3-a4ae-957af1c2c152 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:41.022114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030d0db-dab0-41e3-a4ae-957af1c2c152 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:41.024840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae2f1d-bc6d-4a2b-a026-9b04512a4ff7 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:41.042158 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 265/539] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:41.043976 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9a01e-1c65-4413-86f8-22da5f82a611 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:41.044542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9a01e-1c65-4413-86f8-22da5f82a611 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:41.044979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9a01e-1c65-4413-86f8-22da5f82a611 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:41.051345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 266/540] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:41.052541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9a01e-1c65-4413-86f8-22da5f82a611 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:41.065799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fed57-0dbc-44c7-9f0b-61e3d9633b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:41.066515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fed57-0dbc-44c7-9f0b-61e3d9633b93 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:41.066887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fed57-0dbc-44c7-9f0b-61e3d9633b93 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:41.067250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fed57-0dbc-44c7-9f0b-61e3d9633b93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:41.068321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fed57-0dbc-44c7-9f0b-61e3d9633b93 None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=1a9b29546bc848ef9e92b3d267da18ca) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:41.068470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fed57-0dbc-44c7-9f0b-61e3d9633b93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:41.070095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fed57-0dbc-44c7-9f0b-61e3d9633b93 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:41.076222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 275/541] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:40 2022] GET /identity/v3/role_assignments?user.id=1a9b29546bc848ef9e92b3d267da18ca&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:41.077746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4d945-5740-445f-afc2-ee48f50f1cfa None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:41.078670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4d945-5740-445f-afc2-ee48f50f1cfa None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:41.079090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4d945-5740-445f-afc2-ee48f50f1cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:41.102531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-05f4d945-5740-445f-afc2-ee48f50f1cfa None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:41.102531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-05f4d945-5740-445f-afc2-ee48f50f1cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:41.213359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9a01e-1c65-4413-86f8-22da5f82a611 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:41.236766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 276/542] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:41.271562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd3dc1-7059-4428-ae14-0f4766fbbc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:41.272456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd3dc1-7059-4428-ae14-0f4766fbbc15 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:41.272899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd3dc1-7059-4428-ae14-0f4766fbbc15 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:41.273323 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd3dc1-7059-4428-ae14-0f4766fbbc15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:41.274209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd3dc1-7059-4428-ae14-0f4766fbbc15 None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=00f209d8426a49acaca7e9d00a1b9ce1) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:41.274667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd3dc1-7059-4428-ae14-0f4766fbbc15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:41.276304 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd3dc1-7059-4428-ae14-0f4766fbbc15 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:41.282609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 267/543] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:41 2022] GET /identity/v3/role_assignments?user.id=00f209d8426a49acaca7e9d00a1b9ce1&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:41.532277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4d945-5740-445f-afc2-ee48f50f1cfa None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:41.541291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:41.671859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4d945-5740-445f-afc2-ee48f50f1cfa None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:41.722428 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f100ed-882f-4c84-b460-c90bad533b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:41.723475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f100ed-882f-4c84-b460-c90bad533b02 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:41.723984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f100ed-882f-4c84-b460-c90bad533b02 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:41.724459 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f100ed-882f-4c84-b460-c90bad533b02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:41.741392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f100ed-882f-4c84-b460-c90bad533b02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-a3f100ed-882f-4c84-b460-c90bad533b02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:41.746622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:41.747499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:41.748340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:41.748340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:41.748340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:41.748340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:41.748340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:41.748340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:41.748340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:41.748340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:41.748340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:41.748340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:41.748340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:41.748340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:41.750951 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 277/544] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:41.757298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd3dc1-7059-4428-ae14-0f4766fbbc15 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:41.815222 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:41.867829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd3dc1-7059-4428-ae14-0f4766fbbc15 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:41.910793 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05d9dd-335d-47ab-a586-b2c52b693ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:41.911400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05d9dd-335d-47ab-a586-b2c52b693ae1 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:41.911687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05d9dd-335d-47ab-a586-b2c52b693ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:41.911965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05d9dd-335d-47ab-a586-b2c52b693ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:41.912630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05d9dd-335d-47ab-a586-b2c52b693ae1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:41.912926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05d9dd-335d-47ab-a586-b2c52b693ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:41.914456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05d9dd-335d-47ab-a586-b2c52b693ae1 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:41.928933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 268/545] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:41.934451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f100ed-882f-4c84-b460-c90bad533b02 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:42.052200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:42.082663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a19a5-a353-4485-a0d0-5c03d499f6fa None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:42.082849 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a19a5-a353-4485-a0d0-5c03d499f6fa None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:42.083123 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a19a5-a353-4485-a0d0-5c03d499f6fa None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:42.084750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 269/546] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:38:42.089871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1cfcb-72dd-4c19-b7b1-8d1448f61eea None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:42.090386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1cfcb-72dd-4c19-b7b1-8d1448f61eea None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:42.090877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1cfcb-72dd-4c19-b7b1-8d1448f61eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:42.105823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f100ed-882f-4c84-b460-c90bad533b02 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:42.109887 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-28e1cfcb-72dd-4c19-b7b1-8d1448f61eea None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:42.111350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-28e1cfcb-72dd-4c19-b7b1-8d1448f61eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:42.176917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a95d3d-e80e-474b-9a54-bfcd753f287a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:42.177955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a95d3d-e80e-474b-9a54-bfcd753f287a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:42.178435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a95d3d-e80e-474b-9a54-bfcd753f287a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:42.178921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a95d3d-e80e-474b-9a54-bfcd753f287a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:42.199916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a95d3d-e80e-474b-9a54-bfcd753f287a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:42.202581 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a95d3d-e80e-474b-9a54-bfcd753f287a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:42.220963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 270/547] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-49a95d3d-e80e-474b-9a54-bfcd753f287a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:42.237019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:42.237910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:42.238755 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:42.238755 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:42.238755 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:42.238755 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 278/548] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:42.244186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1cfcb-72dd-4c19-b7b1-8d1448f61eea None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:42.303296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:42.380784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1cfcb-72dd-4c19-b7b1-8d1448f61eea None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:42.419332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475d039-8bdb-4866-8569-1b81158f6bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:42.420287 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475d039-8bdb-4866-8569-1b81158f6bf0 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:42.420791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475d039-8bdb-4866-8569-1b81158f6bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:42.421262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475d039-8bdb-4866-8569-1b81158f6bf0 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:42.422127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475d039-8bdb-4866-8569-1b81158f6bf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:42.422627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475d039-8bdb-4866-8569-1b81158f6bf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:42.424248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475d039-8bdb-4866-8569-1b81158f6bf0 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:42.436247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 271/549] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:42.442311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a95d3d-e80e-474b-9a54-bfcd753f287a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:42.605387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475d039-8bdb-4866-8569-1b81158f6bf0 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:42.621302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a95d3d-e80e-474b-9a54-bfcd753f287a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:42.643659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:42.665183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def095aa-5213-4c4a-9bea-c68f6c1ae45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:42.665814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def095aa-5213-4c4a-9bea-c68f6c1ae45d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:42.666088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def095aa-5213-4c4a-9bea-c68f6c1ae45d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:42.666415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def095aa-5213-4c4a-9bea-c68f6c1ae45d None admin] PATH_INFO: `/v3/users/swift` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:42.676063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def095aa-5213-4c4a-9bea-c68f6c1ae45d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:42.678203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def095aa-5213-4c4a-9bea-c68f6c1ae45d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-def095aa-5213-4c4a-9bea-c68f6c1ae45d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:42.693374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:42.695003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:42.696877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:42.696877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:42.696877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:42.696877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:42.696877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:42.696877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:42.696877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:42.696877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:42.696877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 14:38:42.696877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:42.699206 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 279/550] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:38:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:42.701078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f422d-7ba3-4f06-a148-30f494be4907 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:42.701376 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f422d-7ba3-4f06-a148-30f494be4907 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:42.701656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f422d-7ba3-4f06-a148-30f494be4907 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:42.702931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 280/551] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:42.707711 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f422d-7ba3-4f06-a148-30f494be4907 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:42.735964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475d039-8bdb-4866-8569-1b81158f6bf0 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:42.750866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:42.788329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07a18c-5816-42b6-8f98-85857fc52793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:42.790209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07a18c-5816-42b6-8f98-85857fc52793 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:42.790847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07a18c-5816-42b6-8f98-85857fc52793 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:42.791489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07a18c-5816-42b6-8f98-85857fc52793 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:42.807431 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07a18c-5816-42b6-8f98-85857fc52793 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-9f07a18c-5816-42b6-8f98-85857fc52793 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:42.816154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:42.817460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:42.818655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:42.818655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:42.818655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:42.818655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:42.818655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:42.818655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:42.818655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:42.818655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:42.818655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:42.818655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:42.818655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:42.818655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:42.822446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 272/552] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:42.824390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87e4ab-47ce-41f8-b6fd-47d44fea0fe6 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:42.825081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87e4ab-47ce-41f8-b6fd-47d44fea0fe6 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:42.825695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87e4ab-47ce-41f8-b6fd-47d44fea0fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:42.830810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f422d-7ba3-4f06-a148-30f494be4907 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:42.848544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4e87e4ab-47ce-41f8-b6fd-47d44fea0fe6 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:42.850131 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4e87e4ab-47ce-41f8-b6fd-47d44fea0fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:42.867620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7b7b4-2e21-41e4-968c-a8603ea150ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:42.868255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7b7b4-2e21-41e4-968c-a8603ea150ce None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:42.868540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7b7b4-2e21-41e4-968c-a8603ea150ce None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:42.868813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7b7b4-2e21-41e4-968c-a8603ea150ce None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:42.869539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7b7b4-2e21-41e4-968c-a8603ea150ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:42.869842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7b7b4-2e21-41e4-968c-a8603ea150ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:42.871413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7b7b4-2e21-41e4-968c-a8603ea150ce None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:42.893350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 281/553] 10.222.0.36 () {56 vars in 1135 bytes} [Thu Aug 18 14:38:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:42.894260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7b7b4-2e21-41e4-968c-a8603ea150ce None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:42.935704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 273/554] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:42.937261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:42.938545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87e4ab-47ce-41f8-b6fd-47d44fea0fe6 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:42.999709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7b7b4-2e21-41e4-968c-a8603ea150ce None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:43.035030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf16d3f-1881-4f16-bac2-7d3edaa84078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:43.035646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf16d3f-1881-4f16-bac2-7d3edaa84078 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.035970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf16d3f-1881-4f16-bac2-7d3edaa84078 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.036248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf16d3f-1881-4f16-bac2-7d3edaa84078 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.037022 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf16d3f-1881-4f16-bac2-7d3edaa84078 None admin] RBAC: Adding query filter params (name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:43.037216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf16d3f-1881-4f16-bac2-7d3edaa84078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:43.038589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf16d3f-1881-4f16-bac2-7d3edaa84078 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:43.051555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 282/555] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:43.059603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf16d3f-1881-4f16-bac2-7d3edaa84078 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:43.110104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:43.122629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87e4ab-47ce-41f8-b6fd-47d44fea0fe6 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:43.176095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f9c27-5fd6-46ec-aeae-2fe82969ee3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:43.176801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f9c27-5fd6-46ec-aeae-2fe82969ee3e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.177148 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f9c27-5fd6-46ec-aeae-2fe82969ee3e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.177492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f9c27-5fd6-46ec-aeae-2fe82969ee3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.251661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:43.258206 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf16d3f-1881-4f16-bac2-7d3edaa84078 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:43.307300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f9c27-5fd6-46ec-aeae-2fe82969ee3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:43.308917 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f9c27-5fd6-46ec-aeae-2fe82969ee3e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:43.334290 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0acc4-dfb6-4c0b-a605-d33d8295a05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:43.336112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0acc4-dfb6-4c0b-a605-d33d8295a05e None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.336624 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0acc4-dfb6-4c0b-a605-d33d8295a05e None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.338100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0acc4-dfb6-4c0b-a605-d33d8295a05e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.351930 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0acc4-dfb6-4c0b-a605-d33d8295a05e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:43.353614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0acc4-dfb6-4c0b-a605-d33d8295a05e None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-3cb0acc4-dfb6-4c0b-a605-d33d8295a05e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:43.369321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:43.370508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:43.371572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:43.371572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:43.371572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:43.373004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 283/556] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:43.374249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0acc4-dfb6-4c0b-a605-d33d8295a05e None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:43.437240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:43.448860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 274/557] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:43.449916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f9c27-5fd6-46ec-aeae-2fe82969ee3e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:43.510010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0acc4-dfb6-4c0b-a605-d33d8295a05e None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:43.553859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95566c-8496-4221-ab74-e7a0e52844b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:43.555215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95566c-8496-4221-ab74-e7a0e52844b9 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.555901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95566c-8496-4221-ab74-e7a0e52844b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.556312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95566c-8496-4221-ab74-e7a0e52844b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.572138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95566c-8496-4221-ab74-e7a0e52844b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:43.577053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f9c27-5fd6-46ec-aeae-2fe82969ee3e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-1e95566c-8496-4221-ab74-e7a0e52844b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:43.579551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:43.581038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:43.582063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:43.582063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:43.582063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:43.582063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:43.582063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:43.582063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:43.582063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:43.582063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:43.582063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:43.582063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:43.582063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:43.582063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:43.584621 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 284/558] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:43.590220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95566c-8496-4221-ab74-e7a0e52844b9 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:43.603513 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:43.623270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828d32b-a1e3-4584-b5aa-74de67ecb141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:43.624411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828d32b-a1e3-4584-b5aa-74de67ecb141 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.624696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828d32b-a1e3-4584-b5aa-74de67ecb141 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.625006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828d32b-a1e3-4584-b5aa-74de67ecb141 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.625655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828d32b-a1e3-4584-b5aa-74de67ecb141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:43.625960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828d32b-a1e3-4584-b5aa-74de67ecb141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:43.627325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828d32b-a1e3-4584-b5aa-74de67ecb141 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:43.639936 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 275/559] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:43.640947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828d32b-a1e3-4584-b5aa-74de67ecb141 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:43.694791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95566c-8496-4221-ab74-e7a0e52844b9 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:43.695986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:43.729338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f432a-a1da-438d-85f3-187b73bb3d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:43.729967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f432a-a1da-438d-85f3-187b73bb3d3f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.730248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f432a-a1da-438d-85f3-187b73bb3d3f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.730520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f432a-a1da-438d-85f3-187b73bb3d3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.740366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828d32b-a1e3-4584-b5aa-74de67ecb141 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:43.740894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f432a-a1da-438d-85f3-187b73bb3d3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:43.742341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f432a-a1da-438d-85f3-187b73bb3d3f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-4f3f432a-a1da-438d-85f3-187b73bb3d3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:43.756938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:43.758095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:43.759183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:43.759183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:43.759183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:43.759183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 285/560] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:43.759183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf841eb-37ca-4df9-a88f-812bdd8e68c1 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.759532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf841eb-37ca-4df9-a88f-812bdd8e68c1 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.759671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf841eb-37ca-4df9-a88f-812bdd8e68c1 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.760894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 286/561] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:43.761892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152b52c-12c8-426f-97d9-53d9816fce84 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.762165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152b52c-12c8-426f-97d9-53d9816fce84 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.762433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152b52c-12c8-426f-97d9-53d9816fce84 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.763662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 287/562] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:43.764367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152b52c-12c8-426f-97d9-53d9816fce84 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:43.779282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30f5e8-191c-4420-a567-17c4544599e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:43.779993 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30f5e8-191c-4420-a567-17c4544599e1 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.780273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30f5e8-191c-4420-a567-17c4544599e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.780545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30f5e8-191c-4420-a567-17c4544599e1 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.781229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30f5e8-191c-4420-a567-17c4544599e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:43.781532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30f5e8-191c-4420-a567-17c4544599e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:43.782911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30f5e8-191c-4420-a567-17c4544599e1 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:43.790727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 276/563] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:43.791499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30f5e8-191c-4420-a567-17c4544599e1 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:43.830878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:43.868076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152b52c-12c8-426f-97d9-53d9816fce84 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:43.888991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30f5e8-191c-4420-a567-17c4544599e1 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:43.903512 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ebf60-7aa0-4569-b8b9-7868215ad5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:43.904291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ebf60-7aa0-4569-b8b9-7868215ad5e9 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.904660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ebf60-7aa0-4569-b8b9-7868215ad5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.905023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ebf60-7aa0-4569-b8b9-7868215ad5e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.905351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:43.912258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ebf60-7aa0-4569-b8b9-7868215ad5e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:43.913702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ebf60-7aa0-4569-b8b9-7868215ad5e9 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-e71ebf60-7aa0-4569-b8b9-7868215ad5e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:43.924747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:43.925888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:43.927038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:43.927038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:43.927038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:43.927038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:43.927038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:43.927038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:43.927038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:43.927038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:43.927038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:38:43.927038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:43.927038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 288/564] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Aug 18 14:38:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:43.927038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfd579-5a77-4634-b046-59272437f027 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.927850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfd579-5a77-4634-b046-59272437f027 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.927850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfd579-5a77-4634-b046-59272437f027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.928877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa477cb-3174-4fbf-ac89-f731083ecc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:43.929559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa477cb-3174-4fbf-ac89-f731083ecc6d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.929909 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa477cb-3174-4fbf-ac89-f731083ecc6d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.930258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa477cb-3174-4fbf-ac89-f731083ecc6d None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.930974 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa477cb-3174-4fbf-ac89-f731083ecc6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:43.931289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa477cb-3174-4fbf-ac89-f731083ecc6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:43.932676 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa477cb-3174-4fbf-ac89-f731083ecc6d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:43.945168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-5dcfd579-5a77-4634-b046-59272437f027 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:43.946265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5dcfd579-5a77-4634-b046-59272437f027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:43.947596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 277/565] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:43.948810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a283922-be3c-4720-9f16-ed1f22356a9a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:43.949091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a283922-be3c-4720-9f16-ed1f22356a9a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:43.949356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a283922-be3c-4720-9f16-ed1f22356a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:43.966168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-7a283922-be3c-4720-9f16-ed1f22356a9a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:43.967354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7a283922-be3c-4720-9f16-ed1f22356a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:44.011695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 289/566] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:44.012542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfd579-5a77-4634-b046-59272437f027 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:44.051724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 278/567] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:44.051724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a283922-be3c-4720-9f16-ed1f22356a9a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:44.078617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:44.131053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfd579-5a77-4634-b046-59272437f027 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:44.166051 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8938c0-b406-4915-878b-bf998dd98609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:44.167016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8938c0-b406-4915-878b-bf998dd98609 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:44.167500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8938c0-b406-4915-878b-bf998dd98609 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:44.168004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8938c0-b406-4915-878b-bf998dd98609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:44.180191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8938c0-b406-4915-878b-bf998dd98609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:44.182025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8938c0-b406-4915-878b-bf998dd98609 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:44.197555 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a283922-be3c-4720-9f16-ed1f22356a9a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-9d8938c0-b406-4915-878b-bf998dd98609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:44.198164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:44.199075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:44.199977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:44.199977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:44.199977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:44.201743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 290/568] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:44.202695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8938c0-b406-4915-878b-bf998dd98609 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:44.205557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:44.243026 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293e651-89b9-4ea8-9980-5f9823720453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:44.244330 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293e651-89b9-4ea8-9980-5f9823720453 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:44.244826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293e651-89b9-4ea8-9980-5f9823720453 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:44.245159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293e651-89b9-4ea8-9980-5f9823720453 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:44.245957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293e651-89b9-4ea8-9980-5f9823720453 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:44.246323 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293e651-89b9-4ea8-9980-5f9823720453 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:44.247936 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293e651-89b9-4ea8-9980-5f9823720453 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:44.271500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 279/569] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Aug 18 14:38:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:44.272571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293e651-89b9-4ea8-9980-5f9823720453 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:44.325441 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8938c0-b406-4915-878b-bf998dd98609 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:44.329032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:44.363338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d879f2-8ddc-4117-9d58-563d30bee082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:44.363994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d879f2-8ddc-4117-9d58-563d30bee082 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:44.364276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d879f2-8ddc-4117-9d58-563d30bee082 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:44.364546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d879f2-8ddc-4117-9d58-563d30bee082 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:44.371018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d879f2-8ddc-4117-9d58-563d30bee082 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:44.372474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d879f2-8ddc-4117-9d58-563d30bee082 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:44.378560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293e651-89b9-4ea8-9980-5f9823720453 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-55d879f2-8ddc-4117-9d58-563d30bee082 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:44.384017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:44.385047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:44.385943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:44.385943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:44.385943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:44.385943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:44.385943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:44.385943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:44.385943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:44.385943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:44.385943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 291/570] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:44.385943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d879f2-8ddc-4117-9d58-563d30bee082 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:44.413180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e901c-6a80-4d75-98c2-ca1ed324340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:44.413781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e901c-6a80-4d75-98c2-ca1ed324340a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:44.414059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e901c-6a80-4d75-98c2-ca1ed324340a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:44.414331 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e901c-6a80-4d75-98c2-ca1ed324340a None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:44.414967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e901c-6a80-4d75-98c2-ca1ed324340a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:44.415270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e901c-6a80-4d75-98c2-ca1ed324340a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:44.416666 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e901c-6a80-4d75-98c2-ca1ed324340a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:44.427585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 280/571] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:44.428564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e901c-6a80-4d75-98c2-ca1ed324340a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:44.453267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:44.484456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d879f2-8ddc-4117-9d58-563d30bee082 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:44.517366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7f9b5-1268-48a3-8bf6-ae730fb93a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:44.517980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7f9b5-1268-48a3-8bf6-ae730fb93a9b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:44.518256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7f9b5-1268-48a3-8bf6-ae730fb93a9b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:44.518521 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7f9b5-1268-48a3-8bf6-ae730fb93a9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:44.528719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e901c-6a80-4d75-98c2-ca1ed324340a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:44.528855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7f9b5-1268-48a3-8bf6-ae730fb93a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:44.530266 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7f9b5-1268-48a3-8bf6-ae730fb93a9b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-62b7f9b5-1268-48a3-8bf6-ae730fb93a9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:44.543943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:44.545038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:44.545947 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:44.545947 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:44.545947 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:44.545947 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 292/572] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:44.545947 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7f9b5-1268-48a3-8bf6-ae730fb93a9b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:44.563618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:44.564607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c0b5e-19e1-41a9-ad4a-4119fd851c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:44.565132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c0b5e-19e1-41a9-ad4a-4119fd851c1d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:44.565494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c0b5e-19e1-41a9-ad4a-4119fd851c1d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:44.565683 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c0b5e-19e1-41a9-ad4a-4119fd851c1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:44.575585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c0b5e-19e1-41a9-ad4a-4119fd851c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-fc9c0b5e-19e1-41a9-ad4a-4119fd851c1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:44.580787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:44.581920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 281/573] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:44.583029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c0b5e-19e1-41a9-ad4a-4119fd851c1d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:44.648007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7f9b5-1268-48a3-8bf6-ae730fb93a9b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:44.665173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:44.680875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c0b5e-19e1-41a9-ad4a-4119fd851c1d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:44.681246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039696c6-d796-4bb7-a12d-1b982c4a46eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:44.681873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039696c6-d796-4bb7-a12d-1b982c4a46eb None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:44.682155 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039696c6-d796-4bb7-a12d-1b982c4a46eb None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:44.682427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039696c6-d796-4bb7-a12d-1b982c4a46eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:44.692932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039696c6-d796-4bb7-a12d-1b982c4a46eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-039696c6-d796-4bb7-a12d-1b982c4a46eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:44.697569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:44.698672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 293/574] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:44.699510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039696c6-d796-4bb7-a12d-1b982c4a46eb None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:44.717777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679141a3-3f65-4bb2-9594-b79730a9443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:44.718313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679141a3-3f65-4bb2-9594-b79730a9443d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:44.718584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679141a3-3f65-4bb2-9594-b79730a9443d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:44.718855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679141a3-3f65-4bb2-9594-b79730a9443d None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:44.719539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679141a3-3f65-4bb2-9594-b79730a9443d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:44.719883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679141a3-3f65-4bb2-9594-b79730a9443d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:44.721311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679141a3-3f65-4bb2-9594-b79730a9443d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:44.732886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 282/575] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:44.733658 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679141a3-3f65-4bb2-9594-b79730a9443d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:44.774612 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:44.796724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039696c6-d796-4bb7-a12d-1b982c4a46eb None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:44.830261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53883654-58df-4a06-9878-38ec769146cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:44.830884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53883654-58df-4a06-9878-38ec769146cb None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:44.831163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53883654-58df-4a06-9878-38ec769146cb None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:44.831435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53883654-58df-4a06-9878-38ec769146cb None admin] PATH_INFO: `/v3/users/glance` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:44.832312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679141a3-3f65-4bb2-9594-b79730a9443d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:44.838436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53883654-58df-4a06-9878-38ec769146cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:44.839959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53883654-58df-4a06-9878-38ec769146cb None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-53883654-58df-4a06-9878-38ec769146cb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:44.851854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:44.852809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:44.853772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:44.853772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:44.853772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:44.853772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:44.853772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:44.853772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:44.853772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:44.853772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:44.853772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 14:38:44.853772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:44.853772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 294/576] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:44.854365 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53883654-58df-4a06-9878-38ec769146cb None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:44.870045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c52816-88e9-4064-9fe9-cd848fb6f9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:44.870638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c52816-88e9-4064-9fe9-cd848fb6f9eb None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:44.870915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c52816-88e9-4064-9fe9-cd848fb6f9eb None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:44.871181 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c52816-88e9-4064-9fe9-cd848fb6f9eb None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:44.871845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c52816-88e9-4064-9fe9-cd848fb6f9eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:44.872144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c52816-88e9-4064-9fe9-cd848fb6f9eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:44.873767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c52816-88e9-4064-9fe9-cd848fb6f9eb None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:44.884771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 283/577] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:44.885854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c52816-88e9-4064-9fe9-cd848fb6f9eb None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:44.890054 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:44.954607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53883654-58df-4a06-9878-38ec769146cb None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:44.986220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c52816-88e9-4064-9fe9-cd848fb6f9eb None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:44.989037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e1d0a-1cfd-402f-8ae2-e27cd696c74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:44.989628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e1d0a-1cfd-402f-8ae2-e27cd696c74c None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:44.989902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e1d0a-1cfd-402f-8ae2-e27cd696c74c None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:44.990167 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e1d0a-1cfd-402f-8ae2-e27cd696c74c None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:44.990811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e1d0a-1cfd-402f-8ae2-e27cd696c74c None admin] RBAC: Adding query filter params (name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:44.991108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e1d0a-1cfd-402f-8ae2-e27cd696c74c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:44.992490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e1d0a-1cfd-402f-8ae2-e27cd696c74c None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:44.994548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:45.001280 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 295/578] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:45.002059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e1d0a-1cfd-402f-8ae2-e27cd696c74c None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:45.020295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265f9dc-de42-428f-9dfc-5a0c441ba012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:45.020898 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265f9dc-de42-428f-9dfc-5a0c441ba012 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:45.021192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265f9dc-de42-428f-9dfc-5a0c441ba012 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:45.021449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265f9dc-de42-428f-9dfc-5a0c441ba012 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:45.022113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265f9dc-de42-428f-9dfc-5a0c441ba012 None admin] RBAC: Adding query filter params (name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:45.022412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265f9dc-de42-428f-9dfc-5a0c441ba012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:45.023790 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265f9dc-de42-428f-9dfc-5a0c441ba012 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:45.031508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 284/579] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:45.032315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265f9dc-de42-428f-9dfc-5a0c441ba012 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:45.104970 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:45.108067 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e1d0a-1cfd-402f-8ae2-e27cd696c74c None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:45.140405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265f9dc-de42-428f-9dfc-5a0c441ba012 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:45.140614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81b873-d135-4e7b-84a2-f5a045a15ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:45.141207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81b873-d135-4e7b-84a2-f5a045a15ba6 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:45.141481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81b873-d135-4e7b-84a2-f5a045a15ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:45.141749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81b873-d135-4e7b-84a2-f5a045a15ba6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:45.142524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81b873-d135-4e7b-84a2-f5a045a15ba6 None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=f578727b96564c3db6b5a1e6cac109d8) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:45.142832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81b873-d135-4e7b-84a2-f5a045a15ba6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:45.144270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81b873-d135-4e7b-84a2-f5a045a15ba6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:45.149396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 296/580] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:45 2022] GET /identity/v3/role_assignments?user.id=f578727b96564c3db6b5a1e6cac109d8&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:45.150163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81b873-d135-4e7b-84a2-f5a045a15ba6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:45.189899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60461b-b7f9-43b6-9998-0b83f86a132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:45.191030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60461b-b7f9-43b6-9998-0b83f86a132a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:45.191902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60461b-b7f9-43b6-9998-0b83f86a132a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:45.192704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60461b-b7f9-43b6-9998-0b83f86a132a None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:45.193899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60461b-b7f9-43b6-9998-0b83f86a132a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:45.194398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60461b-b7f9-43b6-9998-0b83f86a132a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:45.196804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60461b-b7f9-43b6-9998-0b83f86a132a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:45.219942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 285/581] 10.222.0.36 () {56 vars in 1137 bytes} [Thu Aug 18 14:38:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:45.221226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60461b-b7f9-43b6-9998-0b83f86a132a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:45.233745 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:45.280947 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81b873-d135-4e7b-84a2-f5a045a15ba6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:45.321729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e99af-7c9e-43f1-8da4-a45fa98aae13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:45.322795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e99af-7c9e-43f1-8da4-a45fa98aae13 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:45.323214 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e99af-7c9e-43f1-8da4-a45fa98aae13 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:45.323489 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e99af-7c9e-43f1-8da4-a45fa98aae13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:45.330313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60461b-b7f9-43b6-9998-0b83f86a132a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:45.332208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e99af-7c9e-43f1-8da4-a45fa98aae13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:45.334640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e99af-7c9e-43f1-8da4-a45fa98aae13 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-d95e99af-7c9e-43f1-8da4-a45fa98aae13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:45.350318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:45.351253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:45.352188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:45.352188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:45.352188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:45.352188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:45.352188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:45.352188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:45.352188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:45.352188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:45.352188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 297/582] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:45.352676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e99af-7c9e-43f1-8da4-a45fa98aae13 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:45.361746 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:45.374848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1849e39-2653-4fdb-ba38-c6d688ac1027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:45.375532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1849e39-2653-4fdb-ba38-c6d688ac1027 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:45.375908 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1849e39-2653-4fdb-ba38-c6d688ac1027 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:45.376585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1849e39-2653-4fdb-ba38-c6d688ac1027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:45.387592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1849e39-2653-4fdb-ba38-c6d688ac1027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:45.389085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1849e39-2653-4fdb-ba38-c6d688ac1027 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-a1849e39-2653-4fdb-ba38-c6d688ac1027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:45.404615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:45.405609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:45.407016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:45.407016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:45.407016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:45.407016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 286/583] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:45.407796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1849e39-2653-4fdb-ba38-c6d688ac1027 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:45.462569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e99af-7c9e-43f1-8da4-a45fa98aae13 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:45.487003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:45.499313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71278db7-8b3f-4286-a4d0-63608177d34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:45.500002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71278db7-8b3f-4286-a4d0-63608177d34e None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:45.500280 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71278db7-8b3f-4286-a4d0-63608177d34e None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:45.500550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71278db7-8b3f-4286-a4d0-63608177d34e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:45.511395 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71278db7-8b3f-4286-a4d0-63608177d34e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:45.512875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71278db7-8b3f-4286-a4d0-63608177d34e None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:45.518262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1849e39-2653-4fdb-ba38-c6d688ac1027 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-71278db7-8b3f-4286-a4d0-63608177d34e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:45.527587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:45.528738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:45.529635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:45.529635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:45.529635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:45.529635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 298/584] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:45.529635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71278db7-8b3f-4286-a4d0-63608177d34e None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:45.560779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2879bd-d275-4be6-bff5-854a623ebba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:45.561488 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2879bd-d275-4be6-bff5-854a623ebba9 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:45.561775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2879bd-d275-4be6-bff5-854a623ebba9 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:45.562414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2879bd-d275-4be6-bff5-854a623ebba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:45.573583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2879bd-d275-4be6-bff5-854a623ebba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:45.574836 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2879bd-d275-4be6-bff5-854a623ebba9 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-ff2879bd-d275-4be6-bff5-854a623ebba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:45.591213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:45.597166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:45.598150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:45.598150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:45.598150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:45.598150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 287/585] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:45.598150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2879bd-d275-4be6-bff5-854a623ebba9 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:45.627542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:45.645206 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71278db7-8b3f-4286-a4d0-63608177d34e None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:45.679419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254b0cf-b1e4-4b6e-b4f8-a6832f23b5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:45.680164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254b0cf-b1e4-4b6e-b4f8-a6832f23b5f2 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:45.680514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254b0cf-b1e4-4b6e-b4f8-a6832f23b5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:45.680882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254b0cf-b1e4-4b6e-b4f8-a6832f23b5f2 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:45.681638 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254b0cf-b1e4-4b6e-b4f8-a6832f23b5f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:45.682014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254b0cf-b1e4-4b6e-b4f8-a6832f23b5f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:45.683512 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254b0cf-b1e4-4b6e-b4f8-a6832f23b5f2 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:45.700350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 299/586] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:45.700350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254b0cf-b1e4-4b6e-b4f8-a6832f23b5f2 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:45.714244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2879bd-d275-4be6-bff5-854a623ebba9 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:45.730931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:45.751009 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23420-9183-4625-8e50-b54b01b76e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:45.751619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23420-9183-4625-8e50-b54b01b76e9e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:45.751929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23420-9183-4625-8e50-b54b01b76e9e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:45.752206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23420-9183-4625-8e50-b54b01b76e9e None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:45.753276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23420-9183-4625-8e50-b54b01b76e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:45.753790 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23420-9183-4625-8e50-b54b01b76e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:45.755121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23420-9183-4625-8e50-b54b01b76e9e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:45.768668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 288/587] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:45.769316 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23420-9183-4625-8e50-b54b01b76e9e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:45.807217 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254b0cf-b1e4-4b6e-b4f8-a6832f23b5f2 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:45.840927 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d1605-d3d4-497c-a734-4886c2074dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:45.841463 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d1605-d3d4-497c-a734-4886c2074dea None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:45.841743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d1605-d3d4-497c-a734-4886c2074dea None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:45.842016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d1605-d3d4-497c-a734-4886c2074dea None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:45.842716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d1605-d3d4-497c-a734-4886c2074dea None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:45.843023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d1605-d3d4-497c-a734-4886c2074dea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:45.844924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d1605-d3d4-497c-a734-4886c2074dea None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:45.861086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 300/588] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:45.861086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d1605-d3d4-497c-a734-4886c2074dea None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:45.866981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:45.882569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23420-9183-4625-8e50-b54b01b76e9e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:45.925190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac8266-8476-4bbe-ad32-c1d9eda95a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:45.925932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac8266-8476-4bbe-ad32-c1d9eda95a7e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:45.926220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac8266-8476-4bbe-ad32-c1d9eda95a7e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:45.926614 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac8266-8476-4bbe-ad32-c1d9eda95a7e None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:45.927429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac8266-8476-4bbe-ad32-c1d9eda95a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:45.927718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac8266-8476-4bbe-ad32-c1d9eda95a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:45.929098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac8266-8476-4bbe-ad32-c1d9eda95a7e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:45.942659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 289/589] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:45.944226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac8266-8476-4bbe-ad32-c1d9eda95a7e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:45.962428 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:45.971064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d1605-d3d4-497c-a734-4886c2074dea None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:46.008025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e9144-c4df-4444-bf83-acfa0992c4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:46.008682 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e9144-c4df-4444-bf83-acfa0992c4c3 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:46.008963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e9144-c4df-4444-bf83-acfa0992c4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:46.009235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e9144-c4df-4444-bf83-acfa0992c4c3 None admin] PATH_INFO: `/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/3f3efa9f3eb24864943b68d241435817/roles/fb5e46782bfc4b09b89697a6b033fecc` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:46.026023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e9144-c4df-4444-bf83-acfa0992c4c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=12f9875f45a84e92a848b8fc1636d6c6, user_id=3f3efa9f3eb24864943b68d241435817, role_id=fb5e46782bfc4b09b89697a6b033fecc)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:46.027470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e9144-c4df-4444-bf83-acfa0992c4c3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:46.043489 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 301/590] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:38:45 2022] PUT /identity/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/3f3efa9f3eb24864943b68d241435817/roles/fb5e46782bfc4b09b89697a6b033fecc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:46.044327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e9144-c4df-4444-bf83-acfa0992c4c3 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:46.055998 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac8266-8476-4bbe-ad32-c1d9eda95a7e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:46.104973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222339c-202b-4787-ae43-bc0470e4c261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:46.107720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222339c-202b-4787-ae43-bc0470e4c261 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:46.107720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222339c-202b-4787-ae43-bc0470e4c261 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:46.107720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222339c-202b-4787-ae43-bc0470e4c261 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:46.116818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222339c-202b-4787-ae43-bc0470e4c261 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:46.118520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222339c-202b-4787-ae43-bc0470e4c261 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-a222339c-202b-4787-ae43-bc0470e4c261 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:46.136965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:46.137985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:46.138932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:46.138932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:46.138932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:46.138932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:46.138932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:46.138932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:46.138932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:46.138932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:46.138932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 14:38:46.138932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:46.140006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 290/591] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:46.140880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222339c-202b-4787-ae43-bc0470e4c261 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:46.158416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:46.192488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e9144-c4df-4444-bf83-acfa0992c4c3 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:46.236953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f3f98-5c71-43cd-9f34-6135dfdc353a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:46.237606 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f3f98-5c71-43cd-9f34-6135dfdc353a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:46.237985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f3f98-5c71-43cd-9f34-6135dfdc353a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:46.238261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f3f98-5c71-43cd-9f34-6135dfdc353a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:46.247589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f3f98-5c71-43cd-9f34-6135dfdc353a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:46.249430 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f3f98-5c71-43cd-9f34-6135dfdc353a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:46.257153 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222339c-202b-4787-ae43-bc0470e4c261 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-8f7f3f98-5c71-43cd-9f34-6135dfdc353a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:46.262492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:46.263421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:46.264417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:46.264417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:46.264417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:46.264417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:46.264417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:46.264417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:46.264417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:46.264417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:46.264417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 14:38:46.264417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:46.264417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 302/592] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:46.265020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f3f98-5c71-43cd-9f34-6135dfdc353a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:46.271822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:46.294495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee1b1e-4192-478c-9462-fba96b6dbc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:46.295122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee1b1e-4192-478c-9462-fba96b6dbc0e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:46.295406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee1b1e-4192-478c-9462-fba96b6dbc0e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:46.295662 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee1b1e-4192-478c-9462-fba96b6dbc0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:46.303221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee1b1e-4192-478c-9462-fba96b6dbc0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:46.304705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee1b1e-4192-478c-9462-fba96b6dbc0e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-96ee1b1e-4192-478c-9462-fba96b6dbc0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:46.315977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:46.317673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:46.319334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:46.319334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:46.319334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:46.319334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:46.319334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:46.319334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:46.319334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:46.319334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:46.319334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 291/593] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:46.319334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee1b1e-4192-478c-9462-fba96b6dbc0e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:46.388974 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:46.391121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f3f98-5c71-43cd-9f34-6135dfdc353a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:46.425706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee1b1e-4192-478c-9462-fba96b6dbc0e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:46.432234 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b82d2-1808-46b1-8aee-a79b6f4e7f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:46.435885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b82d2-1808-46b1-8aee-a79b6f4e7f3a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:46.436212 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b82d2-1808-46b1-8aee-a79b6f4e7f3a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:46.436497 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b82d2-1808-46b1-8aee-a79b6f4e7f3a None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:46.437253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b82d2-1808-46b1-8aee-a79b6f4e7f3a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:46.437628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b82d2-1808-46b1-8aee-a79b6f4e7f3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:46.439197 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b82d2-1808-46b1-8aee-a79b6f4e7f3a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:46.475512 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 303/594] 10.222.0.36 () {56 vars in 1137 bytes} [Thu Aug 18 14:38:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:46.476594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b82d2-1808-46b1-8aee-a79b6f4e7f3a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:46.477739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18349ba-09c4-4510-afed-c9d7e2793f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:46.477739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18349ba-09c4-4510-afed-c9d7e2793f0c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:46.477739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18349ba-09c4-4510-afed-c9d7e2793f0c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:46.477992 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18349ba-09c4-4510-afed-c9d7e2793f0c None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:46.478889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18349ba-09c4-4510-afed-c9d7e2793f0c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:46.479296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18349ba-09c4-4510-afed-c9d7e2793f0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:46.485265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18349ba-09c4-4510-afed-c9d7e2793f0c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:46.509449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 292/595] 10.222.0.36 () {56 vars in 1139 bytes} [Thu Aug 18 14:38:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:46.510426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18349ba-09c4-4510-afed-c9d7e2793f0c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:46.566153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:46.611441 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b82d2-1808-46b1-8aee-a79b6f4e7f3a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:46.629771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18349ba-09c4-4510-afed-c9d7e2793f0c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:46.651167 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12275f-8756-4689-91aa-a54b36ef7b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:46.652084 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12275f-8756-4689-91aa-a54b36ef7b72 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:46.652515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12275f-8756-4689-91aa-a54b36ef7b72 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:46.653158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12275f-8756-4689-91aa-a54b36ef7b72 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:46.654003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12275f-8756-4689-91aa-a54b36ef7b72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:46.654454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12275f-8756-4689-91aa-a54b36ef7b72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:46.656062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12275f-8756-4689-91aa-a54b36ef7b72 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:46.674840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 304/596] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:46.676000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12275f-8756-4689-91aa-a54b36ef7b72 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:46.686072 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:46.702921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae4504-a88e-483f-908d-bdcc17c734c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:46.703344 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae4504-a88e-483f-908d-bdcc17c734c5 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:46.703625 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae4504-a88e-483f-908d-bdcc17c734c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:46.703928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae4504-a88e-483f-908d-bdcc17c734c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:46.723876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae4504-a88e-483f-908d-bdcc17c734c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:46.725037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae4504-a88e-483f-908d-bdcc17c734c5 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-edae4504-a88e-483f-908d-bdcc17c734c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:46.761365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:46.762388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:46.763311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:46.763311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:46.763311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:46.763311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 293/597] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:46.763311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae4504-a88e-483f-908d-bdcc17c734c5 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:46.792617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12275f-8756-4689-91aa-a54b36ef7b72 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:46.817437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:46.831803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30973fc0-bb14-4960-8256-a5505e3470fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:46.832740 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30973fc0-bb14-4960-8256-a5505e3470fe None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:46.833223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30973fc0-bb14-4960-8256-a5505e3470fe None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:46.833698 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30973fc0-bb14-4960-8256-a5505e3470fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:46.845322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30973fc0-bb14-4960-8256-a5505e3470fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:46.847060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30973fc0-bb14-4960-8256-a5505e3470fe None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-30973fc0-bb14-4960-8256-a5505e3470fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:46.862460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:46.863434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:46.864373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:46.864373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:46.864373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:46.866152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 305/598] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:46.867197 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30973fc0-bb14-4960-8256-a5505e3470fe None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:46.934351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae4504-a88e-483f-908d-bdcc17c734c5 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:46.981375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af897871-8cef-408c-97c8-194a953b13d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:46.982037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af897871-8cef-408c-97c8-194a953b13d2 None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:46.982312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af897871-8cef-408c-97c8-194a953b13d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:46.982819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af897871-8cef-408c-97c8-194a953b13d2 None admin] PATH_INFO: `/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/5f2b6dc3406f46e68c74677f5fc7e3b1/roles/fb5e46782bfc4b09b89697a6b033fecc` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:46.989696 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30973fc0-bb14-4960-8256-a5505e3470fe None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:46.999766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af897871-8cef-408c-97c8-194a953b13d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=12f9875f45a84e92a848b8fc1636d6c6, user_id=5f2b6dc3406f46e68c74677f5fc7e3b1, role_id=fb5e46782bfc4b09b89697a6b033fecc)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:47.002266 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af897871-8cef-408c-97c8-194a953b13d2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:47.039731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 294/599] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:38:46 2022] PUT /identity/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/5f2b6dc3406f46e68c74677f5fc7e3b1/roles/fb5e46782bfc4b09b89697a6b033fecc => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:47.039731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af897871-8cef-408c-97c8-194a953b13d2 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:47.056665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c910ff-1077-4ade-9d04-fc777b842311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:47.057315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c910ff-1077-4ade-9d04-fc777b842311 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:47.057597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c910ff-1077-4ade-9d04-fc777b842311 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:47.057874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c910ff-1077-4ade-9d04-fc777b842311 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:47.058542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c910ff-1077-4ade-9d04-fc777b842311 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:47.058851 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c910ff-1077-4ade-9d04-fc777b842311 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:47.060322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c910ff-1077-4ade-9d04-fc777b842311 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:47.107294 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 306/600] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:47.116980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c910ff-1077-4ade-9d04-fc777b842311 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:47.153724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:47.200415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:47.269354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af897871-8cef-408c-97c8-194a953b13d2 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:47.287925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c910ff-1077-4ade-9d04-fc777b842311 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:47.318837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d702f59-6ba5-4372-838e-335b72288091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:47.319446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d702f59-6ba5-4372-838e-335b72288091 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:47.319751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d702f59-6ba5-4372-838e-335b72288091 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:47.320027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d702f59-6ba5-4372-838e-335b72288091 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:47.320724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d702f59-6ba5-4372-838e-335b72288091 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:47.321031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d702f59-6ba5-4372-838e-335b72288091 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:47.322409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d702f59-6ba5-4372-838e-335b72288091 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:47.322633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6efa3-d52b-4c64-8159-5ec1760d4c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:47.323415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6efa3-d52b-4c64-8159-5ec1760d4c80 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:47.323869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6efa3-d52b-4c64-8159-5ec1760d4c80 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:47.324292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6efa3-d52b-4c64-8159-5ec1760d4c80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:47.331304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6efa3-d52b-4c64-8159-5ec1760d4c80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:47.332970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6efa3-d52b-4c64-8159-5ec1760d4c80 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:47.336162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 295/601] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:47.337562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee01e6-658d-44ea-bca7-4d3d7d7fce13 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:47.337846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee01e6-658d-44ea-bca7-4d3d7d7fce13 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:47.338220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee01e6-658d-44ea-bca7-4d3d7d7fce13 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:47.343450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 296/602] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:47.343450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee01e6-658d-44ea-bca7-4d3d7d7fce13 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-f7d6efa3-d52b-4c64-8159-5ec1760d4c80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:47.352369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:47.353467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:47.354636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:47.354636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:47.354636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:47.354636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:47.354636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:47.354636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:47.354636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:47.354636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:47.354636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 307/603] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:47.361115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168402-bc32-457c-8469-86d3bf243d84 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:47.361401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168402-bc32-457c-8469-86d3bf243d84 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:47.361676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168402-bc32-457c-8469-86d3bf243d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:47.384342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-6f168402-bc32-457c-8469-86d3bf243d84 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:47.385757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6f168402-bc32-457c-8469-86d3bf243d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:47.462610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:47.473639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 308/604] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:47.473639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f168402-bc32-457c-8469-86d3bf243d84 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:47.490311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee01e6-658d-44ea-bca7-4d3d7d7fce13 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:47.514579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:47.557684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87e323-3551-4fc4-aaeb-0a74917f1d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:47.558320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87e323-3551-4fc4-aaeb-0a74917f1d10 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:47.558594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87e323-3551-4fc4-aaeb-0a74917f1d10 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:47.558863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87e323-3551-4fc4-aaeb-0a74917f1d10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:47.574397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87e323-3551-4fc4-aaeb-0a74917f1d10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:47.577068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87e323-3551-4fc4-aaeb-0a74917f1d10 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-4b87e323-3551-4fc4-aaeb-0a74917f1d10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:47.598102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:47.599213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:47.600377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:47.600377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:47.600377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:47.600377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:47.600377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:47.600377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:47.600377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:47.600377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:47.600377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 297/605] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:47.612196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87e323-3551-4fc4-aaeb-0a74917f1d10 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:47.630368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f168402-bc32-457c-8469-86d3bf243d84 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:47.669955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f789c-6ffb-46e1-8e5e-937cd3423280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:47.670843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f789c-6ffb-46e1-8e5e-937cd3423280 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:47.671126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f789c-6ffb-46e1-8e5e-937cd3423280 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:47.671396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f789c-6ffb-46e1-8e5e-937cd3423280 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:47.672169 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f789c-6ffb-46e1-8e5e-937cd3423280 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:47.672899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f789c-6ffb-46e1-8e5e-937cd3423280 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:47.674528 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f789c-6ffb-46e1-8e5e-937cd3423280 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:47.691170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 309/606] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:47.696885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f789c-6ffb-46e1-8e5e-937cd3423280 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:47.796124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87e323-3551-4fc4-aaeb-0a74917f1d10 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:47.836552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d546ebe-a58a-4bf3-b5cd-eb05c495ab61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:47.837199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d546ebe-a58a-4bf3-b5cd-eb05c495ab61 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:47.837480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d546ebe-a58a-4bf3-b5cd-eb05c495ab61 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:47.837944 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:47.838608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d546ebe-a58a-4bf3-b5cd-eb05c495ab61 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:47.839304 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d546ebe-a58a-4bf3-b5cd-eb05c495ab61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:47.839612 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d546ebe-a58a-4bf3-b5cd-eb05c495ab61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:47.841169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d546ebe-a58a-4bf3-b5cd-eb05c495ab61 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:47.855619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 298/607] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:47.856464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d546ebe-a58a-4bf3-b5cd-eb05c495ab61 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:47.864005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f789c-6ffb-46e1-8e5e-937cd3423280 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:47.872812 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:47.903279 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be6f62-79bd-44a0-9c72-50b6ae7ccb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:47.904014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be6f62-79bd-44a0-9c72-50b6ae7ccb04 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:47.904287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be6f62-79bd-44a0-9c72-50b6ae7ccb04 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:47.904605 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be6f62-79bd-44a0-9c72-50b6ae7ccb04 None admin] PATH_INFO: `/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/1a9b29546bc848ef9e92b3d267da18ca/roles/fb5e46782bfc4b09b89697a6b033fecc` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:47.924236 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be6f62-79bd-44a0-9c72-50b6ae7ccb04 None admin] RBAC: Authorizing `identity:create_grant(project_id=12f9875f45a84e92a848b8fc1636d6c6, user_id=1a9b29546bc848ef9e92b3d267da18ca, role_id=fb5e46782bfc4b09b89697a6b033fecc)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:47.925746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be6f62-79bd-44a0-9c72-50b6ae7ccb04 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:47.945380 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 310/608] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:38:47 2022] PUT /identity/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/1a9b29546bc848ef9e92b3d267da18ca/roles/fb5e46782bfc4b09b89697a6b033fecc => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:47.951565 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be6f62-79bd-44a0-9c72-50b6ae7ccb04 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:47.982471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d546ebe-a58a-4bf3-b5cd-eb05c495ab61 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:48.024806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab37a4e-2239-49e4-b340-e49dc5c28000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:48.025469 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab37a4e-2239-49e4-b340-e49dc5c28000 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:48.025748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab37a4e-2239-49e4-b340-e49dc5c28000 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:48.026020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab37a4e-2239-49e4-b340-e49dc5c28000 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:48.048782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab37a4e-2239-49e4-b340-e49dc5c28000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-3ab37a4e-2239-49e4-b340-e49dc5c28000 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:48.054090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:48.055012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:48.056776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 299/609] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:48.062691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab37a4e-2239-49e4-b340-e49dc5c28000 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:48.129331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be6f62-79bd-44a0-9c72-50b6ae7ccb04 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:48.170376 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185dd4f-845d-4eea-be09-549511d01ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:48.171055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185dd4f-845d-4eea-be09-549511d01ae4 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:48.171337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185dd4f-845d-4eea-be09-549511d01ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:48.171610 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185dd4f-845d-4eea-be09-549511d01ae4 None admin] PATH_INFO: `/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/00f209d8426a49acaca7e9d00a1b9ce1/roles/fb5e46782bfc4b09b89697a6b033fecc` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:48.175152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:48.181558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:48.196437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185dd4f-845d-4eea-be09-549511d01ae4 None admin] RBAC: Authorizing `identity:create_grant(project_id=12f9875f45a84e92a848b8fc1636d6c6, user_id=00f209d8426a49acaca7e9d00a1b9ce1, role_id=fb5e46782bfc4b09b89697a6b033fecc)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:48.198970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185dd4f-845d-4eea-be09-549511d01ae4 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:48.217020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 311/610] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:38:47 2022] PUT /identity/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/00f209d8426a49acaca7e9d00a1b9ce1/roles/fb5e46782bfc4b09b89697a6b033fecc => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:48.219090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485fb23-b08d-4188-ac89-4d1171a78127 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:48.219598 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485fb23-b08d-4188-ac89-4d1171a78127 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:48.220118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485fb23-b08d-4188-ac89-4d1171a78127 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:48.221921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 312/611] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:48.227371 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96287c-0c78-4bfc-8887-ca63be6e4273 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:48.228113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96287c-0c78-4bfc-8887-ca63be6e4273 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:48.228825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab37a4e-2239-49e4-b340-e49dc5c28000 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:48.229312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96287c-0c78-4bfc-8887-ca63be6e4273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:48.275304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-cb96287c-0c78-4bfc-8887-ca63be6e4273 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:48.279255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-cb96287c-0c78-4bfc-8887-ca63be6e4273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:48.306107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4ce5a-4001-442b-8436-0a2691209c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:48.307252 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4ce5a-4001-442b-8436-0a2691209c6b None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:48.308213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4ce5a-4001-442b-8436-0a2691209c6b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:48.308707 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4ce5a-4001-442b-8436-0a2691209c6b None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:48.309622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4ce5a-4001-442b-8436-0a2691209c6b None admin] RBAC: Adding query filter params (name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:48.310138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4ce5a-4001-442b-8436-0a2691209c6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:48.311804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4ce5a-4001-442b-8436-0a2691209c6b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:48.323346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 300/612] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:48.330253 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4ce5a-4001-442b-8436-0a2691209c6b None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:48.414159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 313/613] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:48.475373 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4ce5a-4001-442b-8436-0a2691209c6b None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:48.505541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:48.511509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5a6d2-1e8f-47dc-8910-a65956c310c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:48.512178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5a6d2-1e8f-47dc-8910-a65956c310c7 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:48.512842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5a6d2-1e8f-47dc-8910-a65956c310c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:48.513121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5a6d2-1e8f-47dc-8910-a65956c310c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:48.524691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5a6d2-1e8f-47dc-8910-a65956c310c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:48.526164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5a6d2-1e8f-47dc-8910-a65956c310c7 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-1de5a6d2-1e8f-47dc-8910-a65956c310c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:48.540743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:48.541729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:48.542624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:48.542624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:48.542624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:48.542624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 301/614] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:48.551529 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96287c-0c78-4bfc-8887-ca63be6e4273 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:48.655743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96287c-0c78-4bfc-8887-ca63be6e4273 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:48.670594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:48.699092 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1096b48-f504-4964-bc39-9cb4cd864101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:48.700060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1096b48-f504-4964-bc39-9cb4cd864101 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:48.700521 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1096b48-f504-4964-bc39-9cb4cd864101 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:48.701022 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1096b48-f504-4964-bc39-9cb4cd864101 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:48.701907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1096b48-f504-4964-bc39-9cb4cd864101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:48.702404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1096b48-f504-4964-bc39-9cb4cd864101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:48.704505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1096b48-f504-4964-bc39-9cb4cd864101 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:48.708923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5a6d2-1e8f-47dc-8910-a65956c310c7 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:48.721656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 314/615] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:48.729004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1096b48-f504-4964-bc39-9cb4cd864101 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:48.830099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5a6d2-1e8f-47dc-8910-a65956c310c7 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:48.867623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb614ec5-697f-474f-8c77-c2c1e6349858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:48.868286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb614ec5-697f-474f-8c77-c2c1e6349858 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:48.868571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb614ec5-697f-474f-8c77-c2c1e6349858 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:48.868848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb614ec5-697f-474f-8c77-c2c1e6349858 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:48.881685 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb614ec5-697f-474f-8c77-c2c1e6349858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-fb614ec5-697f-474f-8c77-c2c1e6349858 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:48.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:48.888354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:48.889188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 302/616] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:48.897383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb614ec5-697f-474f-8c77-c2c1e6349858 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:48.939141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1096b48-f504-4964-bc39-9cb4cd864101 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:48.949858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:49.010813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb614ec5-697f-474f-8c77-c2c1e6349858 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:49.048562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984dd1e-02ee-406c-b1f9-69543a337e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:49.049766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984dd1e-02ee-406c-b1f9-69543a337e94 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:49.051367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984dd1e-02ee-406c-b1f9-69543a337e94 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:49.052233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10252b6b-ddd0-43c4-b31e-74da3b7f7b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:49.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984dd1e-02ee-406c-b1f9-69543a337e94 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:49.054830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10252b6b-ddd0-43c4-b31e-74da3b7f7b57 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:49.055785 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10252b6b-ddd0-43c4-b31e-74da3b7f7b57 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:49.057036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10252b6b-ddd0-43c4-b31e-74da3b7f7b57 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:49.057724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10252b6b-ddd0-43c4-b31e-74da3b7f7b57 None admin] RBAC: Adding query filter params (name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:49.058029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10252b6b-ddd0-43c4-b31e-74da3b7f7b57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:49.059400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10252b6b-ddd0-43c4-b31e-74da3b7f7b57 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:49.069033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 303/617] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:49.069896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984dd1e-02ee-406c-b1f9-69543a337e94 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:49.071351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984dd1e-02ee-406c-b1f9-69543a337e94 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:49.078617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10252b6b-ddd0-43c4-b31e-74da3b7f7b57 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-d984dd1e-02ee-406c-b1f9-69543a337e94 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:49.096015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:49.097030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:49.098059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:49.098059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:49.098059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:49.098059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:49.098059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:49.098059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:49.098059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:49.098059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:49.098059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 14:38:49.098059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:49.098605 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 315/618] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:38:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:49.106594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984dd1e-02ee-406c-b1f9-69543a337e94 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:49.179963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:49.264639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10252b6b-ddd0-43c4-b31e-74da3b7f7b57 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:49.264832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984dd1e-02ee-406c-b1f9-69543a337e94 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:49.267509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:49.305359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0d860-1fcc-4edb-aaa9-3fb3d058feaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:49.306483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0d860-1fcc-4edb-aaa9-3fb3d058feaa None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:49.306983 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0d860-1fcc-4edb-aaa9-3fb3d058feaa None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:49.307439 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0d860-1fcc-4edb-aaa9-3fb3d058feaa None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:49.308367 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b92bd-48a6-4c08-af18-33de516e63e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:49.308697 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0d860-1fcc-4edb-aaa9-3fb3d058feaa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:49.309134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b92bd-48a6-4c08-af18-33de516e63e4 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:49.309377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0d860-1fcc-4edb-aaa9-3fb3d058feaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:49.309480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b92bd-48a6-4c08-af18-33de516e63e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:49.309804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b92bd-48a6-4c08-af18-33de516e63e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:49.311476 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0d860-1fcc-4edb-aaa9-3fb3d058feaa None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:49.323557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b92bd-48a6-4c08-af18-33de516e63e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:49.325118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b92bd-48a6-4c08-af18-33de516e63e4 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:49.336909 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 316/619] 10.222.0.36 () {56 vars in 1135 bytes} [Thu Aug 18 14:38:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:49.338553 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231edad-1e36-4b82-b273-697f8b0d172b None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:49.339049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231edad-1e36-4b82-b273-697f8b0d172b None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:49.339503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231edad-1e36-4b82-b273-697f8b0d172b None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:49.341408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 317/620] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-323b92bd-48a6-4c08-af18-33de516e63e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:49.341516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:49.342577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:49.343485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:49.343485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:49.343485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:49.343485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 304/621] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:49.343824 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b92bd-48a6-4c08-af18-33de516e63e4 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:49.349359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231edad-1e36-4b82-b273-697f8b0d172b None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:49.447421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:49.460463 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231edad-1e36-4b82-b273-697f8b0d172b None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:49.473124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b92bd-48a6-4c08-af18-33de516e63e4 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:49.509069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c0589-e082-4aed-9456-6a52193aff41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:49.509954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c0589-e082-4aed-9456-6a52193aff41 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:49.510231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c0589-e082-4aed-9456-6a52193aff41 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:49.510499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c0589-e082-4aed-9456-6a52193aff41 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:49.511137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c0589-e082-4aed-9456-6a52193aff41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:49.511434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c0589-e082-4aed-9456-6a52193aff41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:49.513791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c0589-e082-4aed-9456-6a52193aff41 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:49.518720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796e520-1671-4a8d-9484-2ea6a1b36093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:49.519609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796e520-1671-4a8d-9484-2ea6a1b36093 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:49.520120 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796e520-1671-4a8d-9484-2ea6a1b36093 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:49.520590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796e520-1671-4a8d-9484-2ea6a1b36093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:49.532279 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 318/622] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:49.533240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796e520-1671-4a8d-9484-2ea6a1b36093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:49.533620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c25465-88cc-410e-ae09-3f5a8e9391d5 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:49.533908 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c25465-88cc-410e-ae09-3f5a8e9391d5 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:49.534181 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c25465-88cc-410e-ae09-3f5a8e9391d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:49.535615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796e520-1671-4a8d-9484-2ea6a1b36093 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-2796e520-1671-4a8d-9484-2ea6a1b36093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:49.551622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:49.552684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:49.553610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:49.553610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:49.553610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:49.553784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-34c25465-88cc-410e-ae09-3f5a8e9391d5 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:49.554955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-34c25465-88cc-410e-ae09-3f5a8e9391d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:49.556356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 305/623] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:49.562134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796e520-1671-4a8d-9484-2ea6a1b36093 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:49.574616 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:49.635705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 319/624] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:49.635705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c25465-88cc-410e-ae09-3f5a8e9391d5 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:49.703706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796e520-1671-4a8d-9484-2ea6a1b36093 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:49.712205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:49.736708 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c25465-88cc-410e-ae09-3f5a8e9391d5 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:49.763070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449e24f-1462-4be4-a9fa-215e2d019a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:49.764072 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449e24f-1462-4be4-a9fa-215e2d019a93 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:49.764631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449e24f-1462-4be4-a9fa-215e2d019a93 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:49.765128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449e24f-1462-4be4-a9fa-215e2d019a93 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:49.769585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0047bd5-146a-4e76-a4e8-63fc2c7e886c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:49.770136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0047bd5-146a-4e76-a4e8-63fc2c7e886c None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:49.770411 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0047bd5-146a-4e76-a4e8-63fc2c7e886c None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:49.770681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0047bd5-146a-4e76-a4e8-63fc2c7e886c None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:49.771317 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0047bd5-146a-4e76-a4e8-63fc2c7e886c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:49.771618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0047bd5-146a-4e76-a4e8-63fc2c7e886c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:49.778334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0047bd5-146a-4e76-a4e8-63fc2c7e886c None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:49.781542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449e24f-1462-4be4-a9fa-215e2d019a93 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:49.783249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449e24f-1462-4be4-a9fa-215e2d019a93 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:49.789444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 320/625] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:49.792700 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0047bd5-146a-4e76-a4e8-63fc2c7e886c None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-2449e24f-1462-4be4-a9fa-215e2d019a93 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:49.800661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:49.801631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:49.802587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:49.802587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:49.802587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:49.802587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:49.802587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:49.802587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:49.802587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:49.802587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:49.802587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:38:49.802587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:49.804832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 306/626] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Aug 18 14:38:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:49.810564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449e24f-1462-4be4-a9fa-215e2d019a93 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:49.881655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:49.901257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0047bd5-146a-4e76-a4e8-63fc2c7e886c None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:49.928651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449e24f-1462-4be4-a9fa-215e2d019a93 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:49.944451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0e087-09e2-4f3a-a043-a44469e8c1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:49.945094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0e087-09e2-4f3a-a043-a44469e8c1f0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:49.945409 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0e087-09e2-4f3a-a043-a44469e8c1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:49.945681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0e087-09e2-4f3a-a043-a44469e8c1f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:49.958135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0e087-09e2-4f3a-a043-a44469e8c1f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:49.959570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0e087-09e2-4f3a-a043-a44469e8c1f0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-37a0e087-09e2-4f3a-a043-a44469e8c1f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:49.980533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:49.981468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:49.982443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:49.982443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:49.982443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:49.982443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:49.982443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:49.982443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:49.982443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:49.982443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:49.982443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 321/627] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:49.989563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0e087-09e2-4f3a-a043-a44469e8c1f0 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:49.992234 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:50.012239 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d49bc5-14f9-4d38-88e1-b2055bc4d46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:50.012905 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d49bc5-14f9-4d38-88e1-b2055bc4d46f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.013115 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d49bc5-14f9-4d38-88e1-b2055bc4d46f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.013382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d49bc5-14f9-4d38-88e1-b2055bc4d46f None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.014120 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d49bc5-14f9-4d38-88e1-b2055bc4d46f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:50.014421 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d49bc5-14f9-4d38-88e1-b2055bc4d46f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:50.021105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d49bc5-14f9-4d38-88e1-b2055bc4d46f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:50.046426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 307/628] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Aug 18 14:38:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:50.047861 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d49bc5-14f9-4d38-88e1-b2055bc4d46f None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:50.113728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0e087-09e2-4f3a-a043-a44469e8c1f0 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:50.148350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41963bef-29a9-4809-b581-2a744cbdd909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:50.148964 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41963bef-29a9-4809-b581-2a744cbdd909 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.149236 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41963bef-29a9-4809-b581-2a744cbdd909 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.149518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41963bef-29a9-4809-b581-2a744cbdd909 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.150191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41963bef-29a9-4809-b581-2a744cbdd909 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:50.150488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41963bef-29a9-4809-b581-2a744cbdd909 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:50.152407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41963bef-29a9-4809-b581-2a744cbdd909 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:50.165233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d49bc5-14f9-4d38-88e1-b2055bc4d46f None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:50.172335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 322/629] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:50.172335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41963bef-29a9-4809-b581-2a744cbdd909 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:50.206906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d94fd-f28a-40ce-b1a9-cb6c215c8500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:50.207998 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d94fd-f28a-40ce-b1a9-cb6c215c8500 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.208597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d94fd-f28a-40ce-b1a9-cb6c215c8500 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.209236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d94fd-f28a-40ce-b1a9-cb6c215c8500 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.210096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:50.222775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d94fd-f28a-40ce-b1a9-cb6c215c8500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-9e2d94fd-f28a-40ce-b1a9-cb6c215c8500 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:50.227966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:50.228946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:50.229796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 308/630] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:50.230514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d94fd-f28a-40ce-b1a9-cb6c215c8500 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:50.243804 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:50.285884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41963bef-29a9-4809-b581-2a744cbdd909 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:50.321124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecee9b4-0cdd-4841-8092-9394586c981d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:50.322079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecee9b4-0cdd-4841-8092-9394586c981d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.322596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecee9b4-0cdd-4841-8092-9394586c981d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.322963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecee9b4-0cdd-4841-8092-9394586c981d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.340770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d94fd-f28a-40ce-b1a9-cb6c215c8500 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:50.376024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3ab8f-9f96-4972-8c65-9931a2245ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:50.376673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3ab8f-9f96-4972-8c65-9931a2245ec3 None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.376947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3ab8f-9f96-4972-8c65-9931a2245ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.377216 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3ab8f-9f96-4972-8c65-9931a2245ec3 None admin] PATH_INFO: `/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/f578727b96564c3db6b5a1e6cac109d8/roles/fb5e46782bfc4b09b89697a6b033fecc` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.392413 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3ab8f-9f96-4972-8c65-9931a2245ec3 None admin] RBAC: Authorizing `identity:create_grant(project_id=12f9875f45a84e92a848b8fc1636d6c6, user_id=f578727b96564c3db6b5a1e6cac109d8, role_id=fb5e46782bfc4b09b89697a6b033fecc)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:50.393880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3ab8f-9f96-4972-8c65-9931a2245ec3 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:50.395288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecee9b4-0cdd-4841-8092-9394586c981d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:50.397078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecee9b4-0cdd-4841-8092-9394586c981d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:50.411769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 309/631] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:38:50 2022] PUT /identity/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/f578727b96564c3db6b5a1e6cac109d8/roles/fb5e46782bfc4b09b89697a6b033fecc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:38:50.413225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419ae79-0618-4d11-9f7a-6af6c258f140 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.413511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419ae79-0618-4d11-9f7a-6af6c258f140 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.413830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419ae79-0618-4d11-9f7a-6af6c258f140 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.416336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 310/632] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:50.416336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419ae79-0618-4d11-9f7a-6af6c258f140 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:50.460066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:50.478262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:50.522218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 323/633] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:50.523499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6c4d6-e270-4ede-b756-040429c2b76b None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.523812 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6c4d6-e270-4ede-b756-040429c2b76b None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.524085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6c4d6-e270-4ede-b756-040429c2b76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.542563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419ae79-0618-4d11-9f7a-6af6c258f140 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:50.543010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-aec6c4d6-e270-4ede-b756-040429c2b76b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:50.544258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-aec6c4d6-e270-4ede-b756-040429c2b76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:50.585852 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c333d7-054e-4137-a041-e1918755f151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:50.586460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c333d7-054e-4137-a041-e1918755f151 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.586738 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c333d7-054e-4137-a041-e1918755f151 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.587723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c333d7-054e-4137-a041-e1918755f151 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.587723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c333d7-054e-4137-a041-e1918755f151 None admin] RBAC: Adding query filter params (name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:50.588004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c333d7-054e-4137-a041-e1918755f151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:50.589396 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c333d7-054e-4137-a041-e1918755f151 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:50.598737 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 311/634] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:50.599966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3507f-51e1-4e76-9f12-cc1c3b1298c0 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.600250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3507f-51e1-4e76-9f12-cc1c3b1298c0 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.600522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3507f-51e1-4e76-9f12-cc1c3b1298c0 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.601869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 312/635] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:50.602395 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3507f-51e1-4e76-9f12-cc1c3b1298c0 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:50.625265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 324/636] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:50.626380 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6c4d6-e270-4ede-b756-040429c2b76b None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:50.722589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:50.732888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6c4d6-e270-4ede-b756-040429c2b76b None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:50.733857 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:50.740566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3507f-51e1-4e76-9f12-cc1c3b1298c0 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:50.772749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0adf6b3-a508-4dc9-806e-e471763a18d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:50.773812 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adf6b3-a508-4dc9-806e-e471763a18d4 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.774314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adf6b3-a508-4dc9-806e-e471763a18d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.774842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adf6b3-a508-4dc9-806e-e471763a18d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.785020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe44678-7c44-4265-b9bf-958c4c7240ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:50.785891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe44678-7c44-4265-b9bf-958c4c7240ac None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.786169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe44678-7c44-4265-b9bf-958c4c7240ac None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.786437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe44678-7c44-4265-b9bf-958c4c7240ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.786924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0adf6b3-a508-4dc9-806e-e471763a18d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:50.788762 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0adf6b3-a508-4dc9-806e-e471763a18d4 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:50.799964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe44678-7c44-4265-b9bf-958c4c7240ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:50.801776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe44678-7c44-4265-b9bf-958c4c7240ac None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-b0adf6b3-a508-4dc9-806e-e471763a18d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:50.809487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:50.810429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:50.811391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:50.811391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:50.811391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:50.815827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 325/637] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:50.818222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031dd16-dac1-4e0c-8c5c-20be1026f9cc None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.818502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031dd16-dac1-4e0c-8c5c-20be1026f9cc None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.818772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031dd16-dac1-4e0c-8c5c-20be1026f9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-1fe44678-7c44-4265-b9bf-958c4c7240ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:50.823376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:50.824359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:50.825353 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:50.825353 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:50.825353 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:50.827512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 313/638] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:50.828623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe44678-7c44-4265-b9bf-958c4c7240ac None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:50.845384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a031dd16-dac1-4e0c-8c5c-20be1026f9cc None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:50.847068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a031dd16-dac1-4e0c-8c5c-20be1026f9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:50.934800 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 326/639] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:50.936021 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031dd16-dac1-4e0c-8c5c-20be1026f9cc None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:50.944577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe44678-7c44-4265-b9bf-958c4c7240ac None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:50.979477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1006d6-16b3-4050-9dc3-348b3cf30c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:50.980126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1006d6-16b3-4050-9dc3-348b3cf30c83 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:50.980401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1006d6-16b3-4050-9dc3-348b3cf30c83 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:50.980672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1006d6-16b3-4050-9dc3-348b3cf30c83 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:50.981304 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1006d6-16b3-4050-9dc3-348b3cf30c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:50.981602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1006d6-16b3-4050-9dc3-348b3cf30c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:50.983564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1006d6-16b3-4050-9dc3-348b3cf30c83 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:51.001142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 314/640] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:51.002014 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1006d6-16b3-4050-9dc3-348b3cf30c83 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:51.016138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:51.039032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:51.075081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031dd16-dac1-4e0c-8c5c-20be1026f9cc None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:51.120175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8be62c-7cbe-48c4-bbdc-4119e64eccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:51.120775 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8be62c-7cbe-48c4-bbdc-4119e64eccf8 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:51.121048 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8be62c-7cbe-48c4-bbdc-4119e64eccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:51.121315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8be62c-7cbe-48c4-bbdc-4119e64eccf8 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:51.121943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8be62c-7cbe-48c4-bbdc-4119e64eccf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:51.122244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8be62c-7cbe-48c4-bbdc-4119e64eccf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:51.122768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1006d6-16b3-4050-9dc3-348b3cf30c83 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:51.123882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8be62c-7cbe-48c4-bbdc-4119e64eccf8 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:51.140954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 327/641] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:51.141804 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8be62c-7cbe-48c4-bbdc-4119e64eccf8 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:51.173256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99a2bd-8197-4f73-bd48-e765523765e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:51.174485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99a2bd-8197-4f73-bd48-e765523765e8 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:51.175224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99a2bd-8197-4f73-bd48-e765523765e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:51.176141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99a2bd-8197-4f73-bd48-e765523765e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:51.190128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99a2bd-8197-4f73-bd48-e765523765e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-8a99a2bd-8197-4f73-bd48-e765523765e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:51.196995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:51.197902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:51.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:51.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:51.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:51.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:51.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:51.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:51.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:51.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:51.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:51.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:51.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:51.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:51.202073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 315/642] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:51.202896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99a2bd-8197-4f73-bd48-e765523765e8 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:51.251145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8be62c-7cbe-48c4-bbdc-4119e64eccf8 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:51.279868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:51.287412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f0d58-0281-4056-8328-00cc38488977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:51.288273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f0d58-0281-4056-8328-00cc38488977 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:51.288555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f0d58-0281-4056-8328-00cc38488977 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:51.288833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f0d58-0281-4056-8328-00cc38488977 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:51.291566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:51.299583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f0d58-0281-4056-8328-00cc38488977 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:51.301240 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f0d58-0281-4056-8328-00cc38488977 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-f68f0d58-0281-4056-8328-00cc38488977 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:51.315541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:51.316801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:51.318013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:51.318013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:51.318013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:51.318013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:51.318013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:51.318013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:51.318013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:51.318013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:51.318013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 14:38:51.318013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:51.318694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99a2bd-8197-4f73-bd48-e765523765e8 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:51.318889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 328/643] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:51.320611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f0d58-0281-4056-8328-00cc38488977 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:51.364225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a58893-8367-4a02-9189-e34198a6923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:51.365505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a58893-8367-4a02-9189-e34198a6923e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:51.365972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a58893-8367-4a02-9189-e34198a6923e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:51.366397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a58893-8367-4a02-9189-e34198a6923e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:51.376233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a58893-8367-4a02-9189-e34198a6923e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:51.378352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a58893-8367-4a02-9189-e34198a6923e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-a6a58893-8367-4a02-9189-e34198a6923e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:51.389478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:51.390405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:51.391448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:51.391448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:51.391448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:51.391448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:51.391448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:51.391448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:51.391448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:51.391448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:51.391448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 316/644] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:51.391448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a58893-8367-4a02-9189-e34198a6923e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:51.428668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f0d58-0281-4056-8328-00cc38488977 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:51.465783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575c1c4-f92f-4b48-a31e-0299264aea75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:51.466419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575c1c4-f92f-4b48-a31e-0299264aea75 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:51.466696 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575c1c4-f92f-4b48-a31e-0299264aea75 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:51.466966 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575c1c4-f92f-4b48-a31e-0299264aea75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:51.478480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575c1c4-f92f-4b48-a31e-0299264aea75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-6575c1c4-f92f-4b48-a31e-0299264aea75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:51.483925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:51.485504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 329/645] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:51.486355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575c1c4-f92f-4b48-a31e-0299264aea75 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:51.511703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a58893-8367-4a02-9189-e34198a6923e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:51.541509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:51.544479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:51.549584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183985c-9f81-4b63-acab-950581ff0e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:51.550188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183985c-9f81-4b63-acab-950581ff0e3b None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:51.550479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183985c-9f81-4b63-acab-950581ff0e3b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:51.550754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183985c-9f81-4b63-acab-950581ff0e3b None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:51.551419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183985c-9f81-4b63-acab-950581ff0e3b None admin] RBAC: Adding query filter params (name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:51.551750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183985c-9f81-4b63-acab-950581ff0e3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:51.553206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183985c-9f81-4b63-acab-950581ff0e3b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:51.561406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 317/646] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:51.566964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183985c-9f81-4b63-acab-950581ff0e3b None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:51.596260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575c1c4-f92f-4b48-a31e-0299264aea75 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:51.632267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188f3e4-b49c-406e-abb1-a0e88a23c627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:51.632924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188f3e4-b49c-406e-abb1-a0e88a23c627 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:51.633211 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188f3e4-b49c-406e-abb1-a0e88a23c627 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:51.633485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188f3e4-b49c-406e-abb1-a0e88a23c627 None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:51.634208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188f3e4-b49c-406e-abb1-a0e88a23c627 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:51.634513 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188f3e4-b49c-406e-abb1-a0e88a23c627 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:51.635972 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188f3e4-b49c-406e-abb1-a0e88a23c627 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:51.652564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 330/647] 10.222.0.36 () {56 vars in 1137 bytes} [Thu Aug 18 14:38:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:51.653355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188f3e4-b49c-406e-abb1-a0e88a23c627 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:51.682109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183985c-9f81-4b63-acab-950581ff0e3b None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:51.716785 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef251a-abf6-4cbe-950b-885282637d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:51.717388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef251a-abf6-4cbe-950b-885282637d86 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:51.717684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef251a-abf6-4cbe-950b-885282637d86 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:51.717942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef251a-abf6-4cbe-950b-885282637d86 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:51.718655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef251a-abf6-4cbe-950b-885282637d86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:51.718963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef251a-abf6-4cbe-950b-885282637d86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:51.720492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef251a-abf6-4cbe-950b-885282637d86 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:51.733336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 318/648] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:51.738755 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef251a-abf6-4cbe-950b-885282637d86 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:51.761378 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188f3e4-b49c-406e-abb1-a0e88a23c627 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:51.774688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:51.794056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:51.799933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b7276-e01e-418e-b93c-c8ccae725b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:51.800811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b7276-e01e-418e-b93c-c8ccae725b5d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:51.801487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b7276-e01e-418e-b93c-c8ccae725b5d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:51.801859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b7276-e01e-418e-b93c-c8ccae725b5d None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:51.802851 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b7276-e01e-418e-b93c-c8ccae725b5d None admin] RBAC: Adding query filter params (name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:51.803387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b7276-e01e-418e-b93c-c8ccae725b5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:51.805603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b7276-e01e-418e-b93c-c8ccae725b5d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:51.819709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 331/649] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:51.820566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b7276-e01e-418e-b93c-c8ccae725b5d None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:51.845650 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef251a-abf6-4cbe-950b-885282637d86 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:51.879441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf769d2-57d3-442e-b37a-3bb89ac7c3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:51.880245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf769d2-57d3-442e-b37a-3bb89ac7c3fc None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:51.880615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf769d2-57d3-442e-b37a-3bb89ac7c3fc None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:51.880975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf769d2-57d3-442e-b37a-3bb89ac7c3fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:51.891423 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf769d2-57d3-442e-b37a-3bb89ac7c3fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:51.893035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf769d2-57d3-442e-b37a-3bb89ac7c3fc None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-abf769d2-57d3-442e-b37a-3bb89ac7c3fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:51.908534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:51.909496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:51.910390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:51.910390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:51.910390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:51.910390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 319/650] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:51.911222 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf769d2-57d3-442e-b37a-3bb89ac7c3fc None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:51.941792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b7276-e01e-418e-b93c-c8ccae725b5d None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:51.977034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae5fc1-f68d-45b7-90a6-c90f6448571a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:51.977639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae5fc1-f68d-45b7-90a6-c90f6448571a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:51.977913 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae5fc1-f68d-45b7-90a6-c90f6448571a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:51.978181 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae5fc1-f68d-45b7-90a6-c90f6448571a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:52.022227 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:52.025770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf769d2-57d3-442e-b37a-3bb89ac7c3fc None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:52.030878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:52.050330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae5fc1-f68d-45b7-90a6-c90f6448571a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:52.051811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae5fc1-f68d-45b7-90a6-c90f6448571a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:52.068411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68579760-4294-4993-bc46-092e8d14d825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:52.069601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579760-4294-4993-bc46-092e8d14d825 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:52.070074 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579760-4294-4993-bc46-092e8d14d825 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:52.070531 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579760-4294-4993-bc46-092e8d14d825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:52.071438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68579760-4294-4993-bc46-092e8d14d825 None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=3f3efa9f3eb24864943b68d241435817) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:52.072362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68579760-4294-4993-bc46-092e8d14d825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:52.074624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68579760-4294-4993-bc46-092e8d14d825 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:52.082877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 320/651] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:51 2022] GET /identity/v3/role_assignments?user.id=3f3efa9f3eb24864943b68d241435817&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:52.084061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68579760-4294-4993-bc46-092e8d14d825 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:52.169099 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 332/652] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:52.170027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae5fc1-f68d-45b7-90a6-c90f6448571a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:52.235236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68579760-4294-4993-bc46-092e8d14d825 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:52.256772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:52.292465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c41a3-1ba7-43ed-8885-bb6be04e699c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:52.293092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c41a3-1ba7-43ed-8885-bb6be04e699c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:52.293372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c41a3-1ba7-43ed-8885-bb6be04e699c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:52.293640 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c41a3-1ba7-43ed-8885-bb6be04e699c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:52.297268 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae5fc1-f68d-45b7-90a6-c90f6448571a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:52.310907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c41a3-1ba7-43ed-8885-bb6be04e699c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:52.312966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c41a3-1ba7-43ed-8885-bb6be04e699c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-026c41a3-1ba7-43ed-8885-bb6be04e699c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:52.327119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:52.328274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:52.329173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:52.329173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:52.329173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:52.329173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 321/653] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:52.329401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c41a3-1ba7-43ed-8885-bb6be04e699c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:52.337942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e7f92-2314-45f0-9a79-8206ca7254ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:52.338513 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e7f92-2314-45f0-9a79-8206ca7254ba None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:52.338782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e7f92-2314-45f0-9a79-8206ca7254ba None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:52.339029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e7f92-2314-45f0-9a79-8206ca7254ba None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:52.339665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e7f92-2314-45f0-9a79-8206ca7254ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:52.339994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e7f92-2314-45f0-9a79-8206ca7254ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:52.341416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e7f92-2314-45f0-9a79-8206ca7254ba None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:52.357824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 333/654] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:52.357824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef52bfe-95a2-4e1e-a54f-021182cd8f74 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:52.357824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef52bfe-95a2-4e1e-a54f-021182cd8f74 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:52.357824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef52bfe-95a2-4e1e-a54f-021182cd8f74 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:52.357824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 334/655] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:52.357824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef52bfe-95a2-4e1e-a54f-021182cd8f74 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:52.360107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:52.444757 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c41a3-1ba7-43ed-8885-bb6be04e699c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:52.468551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef52bfe-95a2-4e1e-a54f-021182cd8f74 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:52.482936 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a8b06-e3ce-4467-bc2f-c7d947d864d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:52.483705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a8b06-e3ce-4467-bc2f-c7d947d864d6 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:52.483899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a8b06-e3ce-4467-bc2f-c7d947d864d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:52.484665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a8b06-e3ce-4467-bc2f-c7d947d864d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:52.498000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a8b06-e3ce-4467-bc2f-c7d947d864d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:52.499783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a8b06-e3ce-4467-bc2f-c7d947d864d6 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:52.504858 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9817f-41c8-4356-8fbf-20309d3b9f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:52.505468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9817f-41c8-4356-8fbf-20309d3b9f3f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:52.505750 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9817f-41c8-4356-8fbf-20309d3b9f3f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:52.506024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9817f-41c8-4356-8fbf-20309d3b9f3f None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:52.506726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9817f-41c8-4356-8fbf-20309d3b9f3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:52.507029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9817f-41c8-4356-8fbf-20309d3b9f3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:52.508491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9817f-41c8-4356-8fbf-20309d3b9f3f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:52.509943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-429a8b06-e3ce-4467-bc2f-c7d947d864d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:52.516257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:52.517198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:52.518126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:52.518126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:52.518126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:52.518126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 322/656] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:52.518126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a8b06-e3ce-4467-bc2f-c7d947d864d6 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:52.525544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 335/657] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:52.527009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29787303-bed7-419e-a67f-218aa5fe8ef9 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:52.527296 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29787303-bed7-419e-a67f-218aa5fe8ef9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:52.527568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29787303-bed7-419e-a67f-218aa5fe8ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:52.545219 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-29787303-bed7-419e-a67f-218aa5fe8ef9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:52.546392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-29787303-bed7-419e-a67f-218aa5fe8ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:52.599498 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:52.627739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 336/658] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:52.627739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29787303-bed7-419e-a67f-218aa5fe8ef9 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:52.631179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a8b06-e3ce-4467-bc2f-c7d947d864d6 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:52.678049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fe805-5a64-414f-bc92-11c5c147c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:52.678679 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fe805-5a64-414f-bc92-11c5c147c53c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:52.678962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fe805-5a64-414f-bc92-11c5c147c53c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:52.679238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fe805-5a64-414f-bc92-11c5c147c53c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:52.687233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fe805-5a64-414f-bc92-11c5c147c53c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:52.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fe805-5a64-414f-bc92-11c5c147c53c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-393fe805-5a64-414f-bc92-11c5c147c53c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:52.700100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:52.701029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:52.701960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:52.701960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:52.701960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:52.701960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:52.701960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:52.701960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:52.701960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:52.701960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:52.701960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 14:38:52.701960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:52.705919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 323/659] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:52.706757 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fe805-5a64-414f-bc92-11c5c147c53c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:52.762389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29787303-bed7-419e-a67f-218aa5fe8ef9 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:52.796484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:52.811491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389477c-cd20-40d9-8257-531f4d65ba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:52.811882 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fe805-5a64-414f-bc92-11c5c147c53c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:52.812618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389477c-cd20-40d9-8257-531f4d65ba8b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:52.813131 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389477c-cd20-40d9-8257-531f4d65ba8b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:52.813802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389477c-cd20-40d9-8257-531f4d65ba8b None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:52.815515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389477c-cd20-40d9-8257-531f4d65ba8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:52.816001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389477c-cd20-40d9-8257-531f4d65ba8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:52.817648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389477c-cd20-40d9-8257-531f4d65ba8b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:52.832418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 337/660] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:52.833585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389477c-cd20-40d9-8257-531f4d65ba8b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:52.845017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:52.851670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801c685-c38d-433d-a4d1-3542cfdbf58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:52.852342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801c685-c38d-433d-a4d1-3542cfdbf58b None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:52.852627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801c685-c38d-433d-a4d1-3542cfdbf58b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:52.852981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801c685-c38d-433d-a4d1-3542cfdbf58b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:52.862087 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801c685-c38d-433d-a4d1-3542cfdbf58b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:52.863549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801c685-c38d-433d-a4d1-3542cfdbf58b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-a801c685-c38d-433d-a4d1-3542cfdbf58b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:52.878528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:52.879479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:52.880453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:52.880453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:52.880453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:52.880453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:52.880453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:52.880453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:52.880453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:52.880453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:52.880453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 14:38:52.880453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:52.884322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 324/661] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:52.885153 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801c685-c38d-433d-a4d1-3542cfdbf58b None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:52.959621 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389477c-cd20-40d9-8257-531f4d65ba8b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:52.992409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801c685-c38d-433d-a4d1-3542cfdbf58b None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:52.994293 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fc1be-3ef3-4aca-8554-91c0e827799b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:52.995026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fc1be-3ef3-4aca-8554-91c0e827799b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:52.995358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fc1be-3ef3-4aca-8554-91c0e827799b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:52.995638 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fc1be-3ef3-4aca-8554-91c0e827799b None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:52.996395 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fc1be-3ef3-4aca-8554-91c0e827799b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:52.996703 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fc1be-3ef3-4aca-8554-91c0e827799b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:52.998135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fc1be-3ef3-4aca-8554-91c0e827799b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:53.016858 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 338/662] 10.222.0.36 () {56 vars in 1137 bytes} [Thu Aug 18 14:38:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:53.017691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fc1be-3ef3-4aca-8554-91c0e827799b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:53.030984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316c745-0ead-478a-832f-70343e3177af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:53.032230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316c745-0ead-478a-832f-70343e3177af None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:53.032442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316c745-0ead-478a-832f-70343e3177af None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:53.032710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316c745-0ead-478a-832f-70343e3177af None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:53.039312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316c745-0ead-478a-832f-70343e3177af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:53.040798 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316c745-0ead-478a-832f-70343e3177af None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-e316c745-0ead-478a-832f-70343e3177af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:53.051732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:53.052639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:53.053533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:53.053533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:53.053533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:53.053533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:53.053533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:53.053533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:53.053533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:53.053533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:53.053533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 325/663] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:53.053533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316c745-0ead-478a-832f-70343e3177af None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:53.074041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:53.111347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:53.139346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fc1be-3ef3-4aca-8554-91c0e827799b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:53.176007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316c745-0ead-478a-832f-70343e3177af None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:53.181418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1901d-e232-43d2-aca9-6ee4b6abe333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:53.182030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1901d-e232-43d2-aca9-6ee4b6abe333 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:53.182308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1901d-e232-43d2-aca9-6ee4b6abe333 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:53.182583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1901d-e232-43d2-aca9-6ee4b6abe333 None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:53.183303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1901d-e232-43d2-aca9-6ee4b6abe333 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:53.183615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1901d-e232-43d2-aca9-6ee4b6abe333 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:53.185056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1901d-e232-43d2-aca9-6ee4b6abe333 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:53.201942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 339/664] 10.222.0.36 () {56 vars in 1139 bytes} [Thu Aug 18 14:38:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:53.202729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1901d-e232-43d2-aca9-6ee4b6abe333 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:53.217524 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e756eb-acc1-49e3-8663-5ac116ba0b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:53.218239 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e756eb-acc1-49e3-8663-5ac116ba0b0d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:53.218360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e756eb-acc1-49e3-8663-5ac116ba0b0d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:53.219180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e756eb-acc1-49e3-8663-5ac116ba0b0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:53.232284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e756eb-acc1-49e3-8663-5ac116ba0b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-57e756eb-acc1-49e3-8663-5ac116ba0b0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:53.237797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:53.238801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:53.239637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 326/665] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:38:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:53.240365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e756eb-acc1-49e3-8663-5ac116ba0b0d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:53.316737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:53.320009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1901d-e232-43d2-aca9-6ee4b6abe333 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:53.349430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e756eb-acc1-49e3-8663-5ac116ba0b0d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:53.360229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d32de-0f65-4399-bf4d-c20fb1ea2f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:53.360923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d32de-0f65-4399-bf4d-c20fb1ea2f6b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:53.361176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d32de-0f65-4399-bf4d-c20fb1ea2f6b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:53.361364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d32de-0f65-4399-bf4d-c20fb1ea2f6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:53.362872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:53.388190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f27a7-2a76-4cda-a576-6f0eb5f3f944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:53.388937 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f27a7-2a76-4cda-a576-6f0eb5f3f944 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:53.389212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f27a7-2a76-4cda-a576-6f0eb5f3f944 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:53.389805 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f27a7-2a76-4cda-a576-6f0eb5f3f944 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:53.391042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f27a7-2a76-4cda-a576-6f0eb5f3f944 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:53.391349 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f27a7-2a76-4cda-a576-6f0eb5f3f944 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:53.392939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f27a7-2a76-4cda-a576-6f0eb5f3f944 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:53.409274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 327/666] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:53.409274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f27a7-2a76-4cda-a576-6f0eb5f3f944 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:53.438798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d32de-0f65-4399-bf4d-c20fb1ea2f6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:53.440375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d32de-0f65-4399-bf4d-c20fb1ea2f6b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:53.510149 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f27a7-2a76-4cda-a576-6f0eb5f3f944 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:53.544366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628a474-ea0b-4b4a-aab1-62717bc9ff0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:53.545468 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628a474-ea0b-4b4a-aab1-62717bc9ff0b None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:53.545976 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628a474-ea0b-4b4a-aab1-62717bc9ff0b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:53.546439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628a474-ea0b-4b4a-aab1-62717bc9ff0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:53.546571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:53.556260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 340/667] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:53.557453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d32de-0f65-4399-bf4d-c20fb1ea2f6b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:53.596919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:53.626838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628a474-ea0b-4b4a-aab1-62717bc9ff0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:53.628447 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628a474-ea0b-4b4a-aab1-62717bc9ff0b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:53.655580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d32de-0f65-4399-bf4d-c20fb1ea2f6b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:53.689945 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65572d-c7b4-4778-83f4-c8f14480804f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:53.690818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65572d-c7b4-4778-83f4-c8f14480804f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:53.691270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65572d-c7b4-4778-83f4-c8f14480804f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:53.691718 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65572d-c7b4-4778-83f4-c8f14480804f None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:53.692516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65572d-c7b4-4778-83f4-c8f14480804f None admin] RBAC: Adding query filter params (name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:53.692963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65572d-c7b4-4778-83f4-c8f14480804f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:53.694464 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65572d-c7b4-4778-83f4-c8f14480804f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:53.703829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 341/668] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:38:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:53.704813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65572d-c7b4-4778-83f4-c8f14480804f None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:53.741286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 328/669] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:53.742392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628a474-ea0b-4b4a-aab1-62717bc9ff0b None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:53.768165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:53.810650 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65572d-c7b4-4778-83f4-c8f14480804f None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:53.817282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:53.846433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69c37a-c855-419f-b19e-c5157689e728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:53.847093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69c37a-c855-419f-b19e-c5157689e728 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:53.847369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69c37a-c855-419f-b19e-c5157689e728 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:53.847639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69c37a-c855-419f-b19e-c5157689e728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:53.848391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69c37a-c855-419f-b19e-c5157689e728 None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=5f2b6dc3406f46e68c74677f5fc7e3b1) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:53.848706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69c37a-c855-419f-b19e-c5157689e728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:53.848805 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628a474-ea0b-4b4a-aab1-62717bc9ff0b None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:53.850258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69c37a-c855-419f-b19e-c5157689e728 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:53.860037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 342/670] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:53 2022] GET /identity/v3/role_assignments?user.id=5f2b6dc3406f46e68c74677f5fc7e3b1&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:53.860954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69c37a-c855-419f-b19e-c5157689e728 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:53.891577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215757-552a-433f-955f-db024e9f570d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:53.892743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215757-552a-433f-955f-db024e9f570d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:53.893018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215757-552a-433f-955f-db024e9f570d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:53.893293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215757-552a-433f-955f-db024e9f570d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:53.908426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70215757-552a-433f-955f-db024e9f570d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:53.908426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70215757-552a-433f-955f-db024e9f570d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-70215757-552a-433f-955f-db024e9f570d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:53.930935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:53.931991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:53.932880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:53.932880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:53.932880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:53.932880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 329/671] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:53.932880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215757-552a-433f-955f-db024e9f570d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:53.974377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69c37a-c855-419f-b19e-c5157689e728 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:54.002482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:54.013133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8ae88-66ee-4d24-a6a5-398832a454c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:54.013133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8ae88-66ee-4d24-a6a5-398832a454c7 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:54.013561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8ae88-66ee-4d24-a6a5-398832a454c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:54.015913 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8ae88-66ee-4d24-a6a5-398832a454c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:54.027519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8ae88-66ee-4d24-a6a5-398832a454c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:54.029066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8ae88-66ee-4d24-a6a5-398832a454c7 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-43a8ae88-66ee-4d24-a6a5-398832a454c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:54.048454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:54.049498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:54.050392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:54.050392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:54.050392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:54.050392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 343/672] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:54.050392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8ae88-66ee-4d24-a6a5-398832a454c7 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:54.057705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215757-552a-433f-955f-db024e9f570d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:54.083237 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:54.096800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13fce2-c1ca-4fad-8add-a2eba2f3ceb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:54.097430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13fce2-c1ca-4fad-8add-a2eba2f3ceb2 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:54.097712 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13fce2-c1ca-4fad-8add-a2eba2f3ceb2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:54.097986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13fce2-c1ca-4fad-8add-a2eba2f3ceb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:54.109456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13fce2-c1ca-4fad-8add-a2eba2f3ceb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:54.110906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13fce2-c1ca-4fad-8add-a2eba2f3ceb2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-cd13fce2-c1ca-4fad-8add-a2eba2f3ceb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:54.126301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:54.127277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:54.128782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:54.128782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:54.128782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:54.128782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 330/673] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:54.128782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13fce2-c1ca-4fad-8add-a2eba2f3ceb2 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:54.162590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8ae88-66ee-4d24-a6a5-398832a454c7 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:54.198473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f2af0-c144-4987-9760-752973094bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:54.199113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f2af0-c144-4987-9760-752973094bf7 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:54.199388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f2af0-c144-4987-9760-752973094bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:54.199659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f2af0-c144-4987-9760-752973094bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:54.200331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f2af0-c144-4987-9760-752973094bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:54.200658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f2af0-c144-4987-9760-752973094bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:54.201994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f2af0-c144-4987-9760-752973094bf7 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:54.215147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 344/674] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:54.220776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a87ce7-b8c1-4dd6-a133-c213bcdb1789 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:54.221065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a87ce7-b8c1-4dd6-a133-c213bcdb1789 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:54.221343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a87ce7-b8c1-4dd6-a133-c213bcdb1789 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:54.222558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 345/675] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:54.223250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a87ce7-b8c1-4dd6-a133-c213bcdb1789 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:54.239561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13fce2-c1ca-4fad-8add-a2eba2f3ceb2 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:54.264159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:54.278996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb78a20-6ccc-4a5b-b826-603fb10e69e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:54.279885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb78a20-6ccc-4a5b-b826-603fb10e69e4 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:54.280473 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb78a20-6ccc-4a5b-b826-603fb10e69e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:54.281069 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb78a20-6ccc-4a5b-b826-603fb10e69e4 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:54.281923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb78a20-6ccc-4a5b-b826-603fb10e69e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:54.282271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb78a20-6ccc-4a5b-b826-603fb10e69e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:54.283786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb78a20-6ccc-4a5b-b826-603fb10e69e4 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:54.301304 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 331/676] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:54.301304 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb78a20-6ccc-4a5b-b826-603fb10e69e4 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:54.332459 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a87ce7-b8c1-4dd6-a133-c213bcdb1789 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:54.336908 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:54.367841 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece81b6d-e4ca-4a34-8f0d-d3752819fce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:54.368447 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece81b6d-e4ca-4a34-8f0d-d3752819fce3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:54.368724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece81b6d-e4ca-4a34-8f0d-d3752819fce3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:54.368995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece81b6d-e4ca-4a34-8f0d-d3752819fce3 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:54.369631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece81b6d-e4ca-4a34-8f0d-d3752819fce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:54.369932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece81b6d-e4ca-4a34-8f0d-d3752819fce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:54.371292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece81b6d-e4ca-4a34-8f0d-d3752819fce3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:54.383555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 346/677] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:54.384761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a23eb7-efb6-40bb-b2f5-3a0e6c6f8832 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:54.385045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a23eb7-efb6-40bb-b2f5-3a0e6c6f8832 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:54.385318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a23eb7-efb6-40bb-b2f5-3a0e6c6f8832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:54.402760 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-21a23eb7-efb6-40bb-b2f5-3a0e6c6f8832 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:54.403973 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-21a23eb7-efb6-40bb-b2f5-3a0e6c6f8832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:54.415031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb78a20-6ccc-4a5b-b826-603fb10e69e4 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:54.450102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba9d2d-07ea-4f13-9f80-1a4dccb0d2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:54.450725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba9d2d-07ea-4f13-9f80-1a4dccb0d2ac None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:54.451012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba9d2d-07ea-4f13-9f80-1a4dccb0d2ac None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:54.451281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba9d2d-07ea-4f13-9f80-1a4dccb0d2ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:54.458467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba9d2d-07ea-4f13-9f80-1a4dccb0d2ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:54.459932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba9d2d-07ea-4f13-9f80-1a4dccb0d2ac None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-96ba9d2d-07ea-4f13-9f80-1a4dccb0d2ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:54.471445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:54.472375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:54.473261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:54.473261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:54.473261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:54.473261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:54.473261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:54.473261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:54.473261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:54.473261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:54.473261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 332/678] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:54.473750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba9d2d-07ea-4f13-9f80-1a4dccb0d2ac None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:54.487770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 347/679] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:54.487770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a23eb7-efb6-40bb-b2f5-3a0e6c6f8832 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:54.534277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:54.578636 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:54.585090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba9d2d-07ea-4f13-9f80-1a4dccb0d2ac None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:54.623406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d530329-b581-41d7-b792-6976cd5206aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:54.625473 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d530329-b581-41d7-b792-6976cd5206aa None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:54.625827 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d530329-b581-41d7-b792-6976cd5206aa None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:54.626168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d530329-b581-41d7-b792-6976cd5206aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:54.631431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a23eb7-efb6-40bb-b2f5-3a0e6c6f8832 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:54.635223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d530329-b581-41d7-b792-6976cd5206aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:54.636762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d530329-b581-41d7-b792-6976cd5206aa None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-1d530329-b581-41d7-b792-6976cd5206aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:54.651017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:54.652207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:54.653385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:54.653385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:54.653385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:54.653385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:54.653385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:54.653385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:54.653385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:54.653385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:54.653385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 14:38:54.653385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:54.654767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 333/680] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:38:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:54.655806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d530329-b581-41d7-b792-6976cd5206aa None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:54.679287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e041970-86a1-482f-873c-3442ee3102ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:54.680527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e041970-86a1-482f-873c-3442ee3102ce None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:54.680957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e041970-86a1-482f-873c-3442ee3102ce None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:54.681377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e041970-86a1-482f-873c-3442ee3102ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:54.692120 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e041970-86a1-482f-873c-3442ee3102ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:54.693556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e041970-86a1-482f-873c-3442ee3102ce None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-8e041970-86a1-482f-873c-3442ee3102ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:54.706920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:54.708145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:54.709037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:54.709037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:54.709037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:54.709037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:54.709037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:54.709037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:54.709037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:54.709037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:54.709037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 348/681] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:54.709547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e041970-86a1-482f-873c-3442ee3102ce None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:54.796175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d530329-b581-41d7-b792-6976cd5206aa None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:54.801623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:54.814924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e041970-86a1-482f-873c-3442ee3102ce None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:54.833516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a485ccd-238d-486f-bded-c81a1bc5ddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:54.834124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a485ccd-238d-486f-bded-c81a1bc5ddb9 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:54.834402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a485ccd-238d-486f-bded-c81a1bc5ddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:54.834672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a485ccd-238d-486f-bded-c81a1bc5ddb9 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:54.835359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a485ccd-238d-486f-bded-c81a1bc5ddb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:54.835663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a485ccd-238d-486f-bded-c81a1bc5ddb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:54.837116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a485ccd-238d-486f-bded-c81a1bc5ddb9 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:54.849499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 334/682] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:54.850296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a485ccd-238d-486f-bded-c81a1bc5ddb9 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:54.855075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e65ff-b055-4215-a376-5c38a9177b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:54.855710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e65ff-b055-4215-a376-5c38a9177b9f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:54.855990 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e65ff-b055-4215-a376-5c38a9177b9f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:54.856261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e65ff-b055-4215-a376-5c38a9177b9f None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:54.856986 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e65ff-b055-4215-a376-5c38a9177b9f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:54.857289 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e65ff-b055-4215-a376-5c38a9177b9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:54.858713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e65ff-b055-4215-a376-5c38a9177b9f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:54.872702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:54.880102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 349/683] 10.222.0.36 () {56 vars in 1135 bytes} [Thu Aug 18 14:38:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:54.880102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e65ff-b055-4215-a376-5c38a9177b9f None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:54.966695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a485ccd-238d-486f-bded-c81a1bc5ddb9 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:54.987671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e65ff-b055-4215-a376-5c38a9177b9f None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:55.003216 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adbac4-af01-48ee-966c-9625e24fb5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:55.003851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adbac4-af01-48ee-966c-9625e24fb5b0 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:55.004239 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adbac4-af01-48ee-966c-9625e24fb5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:55.004507 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adbac4-af01-48ee-966c-9625e24fb5b0 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:55.005299 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adbac4-af01-48ee-966c-9625e24fb5b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:55.005599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adbac4-af01-48ee-966c-9625e24fb5b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:55.007008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adbac4-af01-48ee-966c-9625e24fb5b0 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:55.024615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 335/684] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:55.024615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adbac4-af01-48ee-966c-9625e24fb5b0 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:55.026588 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d193c-f368-41e4-a615-df9ab65bf7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:55.027239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d193c-f368-41e4-a615-df9ab65bf7d3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:55.027516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d193c-f368-41e4-a615-df9ab65bf7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:55.027822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d193c-f368-41e4-a615-df9ab65bf7d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:55.039032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d193c-f368-41e4-a615-df9ab65bf7d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-fc1d193c-f368-41e4-a615-df9ab65bf7d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:38:55.043920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:55.044933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:55.045817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 350/685] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:38:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:55.046573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d193c-f368-41e4-a615-df9ab65bf7d3 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:55.057460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:55.113276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:55.139903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adbac4-af01-48ee-966c-9625e24fb5b0 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:55.159917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d193c-f368-41e4-a615-df9ab65bf7d3 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:55.175782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34de16-878c-498f-ab98-ccb6cfd14979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:55.176374 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34de16-878c-498f-ab98-ccb6cfd14979 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:55.176662 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34de16-878c-498f-ab98-ccb6cfd14979 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:55.176921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34de16-878c-498f-ab98-ccb6cfd14979 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:55.177847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34de16-878c-498f-ab98-ccb6cfd14979 None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=1a9b29546bc848ef9e92b3d267da18ca) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:55.177962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34de16-878c-498f-ab98-ccb6cfd14979 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:55.180383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34de16-878c-498f-ab98-ccb6cfd14979 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:55.186685 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 336/686] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:55 2022] GET /identity/v3/role_assignments?user.id=1a9b29546bc848ef9e92b3d267da18ca&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:55.196219 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4000de-e2ad-4d47-8381-0d6c84af8f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:55.196992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4000de-e2ad-4d47-8381-0d6c84af8f02 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:55.197236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34de16-878c-498f-ab98-ccb6cfd14979 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:55.197465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4000de-e2ad-4d47-8381-0d6c84af8f02 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:55.197836 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4000de-e2ad-4d47-8381-0d6c84af8f02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:55.276787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4000de-e2ad-4d47-8381-0d6c84af8f02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:55.278444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4000de-e2ad-4d47-8381-0d6c84af8f02 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:55.288083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:55.348730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34de16-878c-498f-ab98-ccb6cfd14979 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:55.387420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f05b72-f477-4a1b-92c8-58511e93512a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:55.388059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f05b72-f477-4a1b-92c8-58511e93512a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:55.388337 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f05b72-f477-4a1b-92c8-58511e93512a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:55.388613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f05b72-f477-4a1b-92c8-58511e93512a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:55.389343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f05b72-f477-4a1b-92c8-58511e93512a None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=00f209d8426a49acaca7e9d00a1b9ce1) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:55.389657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f05b72-f477-4a1b-92c8-58511e93512a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:55.391125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f05b72-f477-4a1b-92c8-58511e93512a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:55.397338 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 337/687] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:55 2022] GET /identity/v3/role_assignments?user.id=00f209d8426a49acaca7e9d00a1b9ce1&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:55.398021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f05b72-f477-4a1b-92c8-58511e93512a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:55.404028 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 351/688] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:55.409658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4000de-e2ad-4d47-8381-0d6c84af8f02 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:55.412057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:55.539712 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f05b72-f477-4a1b-92c8-58511e93512a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:55.540984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4000de-e2ad-4d47-8381-0d6c84af8f02 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:55.551035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:55.601137 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec93f32-6c44-4a67-999c-0fc9620f763d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:55.601758 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec93f32-6c44-4a67-999c-0fc9620f763d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:55.602035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec93f32-6c44-4a67-999c-0fc9620f763d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:55.602966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec93f32-6c44-4a67-999c-0fc9620f763d None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:55.603074 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec93f32-6c44-4a67-999c-0fc9620f763d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:55.603355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec93f32-6c44-4a67-999c-0fc9620f763d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:55.604054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0bc68-1ce2-481d-b68c-afac96cfad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:55.604770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec93f32-6c44-4a67-999c-0fc9620f763d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:55.605086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0bc68-1ce2-481d-b68c-afac96cfad3a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:55.605531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0bc68-1ce2-481d-b68c-afac96cfad3a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:55.606250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0bc68-1ce2-481d-b68c-afac96cfad3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:55.618948 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 338/689] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:38:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:55.619917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0bc68-1ce2-481d-b68c-afac96cfad3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:55.620106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec93f32-6c44-4a67-999c-0fc9620f763d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:55.621744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0bc68-1ce2-481d-b68c-afac96cfad3a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-bce0bc68-1ce2-481d-b68c-afac96cfad3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:55.642483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:55.643479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:55.644393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:55.644393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:55.644393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:55.645384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 352/690] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:55.650703 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0bc68-1ce2-481d-b68c-afac96cfad3a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:55.705003 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:55.744385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec93f32-6c44-4a67-999c-0fc9620f763d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:55.759626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0bc68-1ce2-481d-b68c-afac96cfad3a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:55.784684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc22d5-bfd7-423e-a5d9-9f36cf1eba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:55.785651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc22d5-bfd7-423e-a5d9-9f36cf1eba91 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:55.786099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc22d5-bfd7-423e-a5d9-9f36cf1eba91 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:55.786613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc22d5-bfd7-423e-a5d9-9f36cf1eba91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:55.797463 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377f19a-4fc2-4bc0-905a-b06d97acc436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:55.798251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377f19a-4fc2-4bc0-905a-b06d97acc436 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:55.798501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc22d5-bfd7-423e-a5d9-9f36cf1eba91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:55.798754 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377f19a-4fc2-4bc0-905a-b06d97acc436 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:55.799138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377f19a-4fc2-4bc0-905a-b06d97acc436 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:55.799957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377f19a-4fc2-4bc0-905a-b06d97acc436 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:55.800355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377f19a-4fc2-4bc0-905a-b06d97acc436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:55.800957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc22d5-bfd7-423e-a5d9-9f36cf1eba91 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:55.802308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377f19a-4fc2-4bc0-905a-b06d97acc436 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:55.816939 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 353/691] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-53cc22d5-bfd7-423e-a5d9-9f36cf1eba91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:55.818179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:55.819112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:55.820389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:55.820389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:55.820389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:55.820389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 339/692] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:55.822150 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377f19a-4fc2-4bc0-905a-b06d97acc436 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:55.825008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc22d5-bfd7-423e-a5d9-9f36cf1eba91 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:55.846751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:55.937400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc22d5-bfd7-423e-a5d9-9f36cf1eba91 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:55.954980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377f19a-4fc2-4bc0-905a-b06d97acc436 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:55.960032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:55.977669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ee47b-882c-434f-88fa-49b6d757bdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:55.978563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ee47b-882c-434f-88fa-49b6d757bdec None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:55.978989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ee47b-882c-434f-88fa-49b6d757bdec None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:55.979411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ee47b-882c-434f-88fa-49b6d757bdec None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:55.980233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ee47b-882c-434f-88fa-49b6d757bdec None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:55.980706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ee47b-882c-434f-88fa-49b6d757bdec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:55.982229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ee47b-882c-434f-88fa-49b6d757bdec None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:55.991987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d4e50-c581-464f-8bcd-7801bde9891b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:55.992613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d4e50-c581-464f-8bcd-7801bde9891b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:55.992888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d4e50-c581-464f-8bcd-7801bde9891b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:55.993163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d4e50-c581-464f-8bcd-7801bde9891b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:55.995673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 340/693] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:38:55.997068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74b742-cb3a-4d66-92c4-5f0e896b7b74 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:55.997501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74b742-cb3a-4d66-92c4-5f0e896b7b74 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:55.997918 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74b742-cb3a-4d66-92c4-5f0e896b7b74 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:55.999656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 341/694] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:56.004642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74b742-cb3a-4d66-92c4-5f0e896b7b74 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:56.004804 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d4e50-c581-464f-8bcd-7801bde9891b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:56.006252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d4e50-c581-464f-8bcd-7801bde9891b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-5b4d4e50-c581-464f-8bcd-7801bde9891b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:56.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:56.019807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:56.020894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:56.020894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:56.020894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:56.020894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:56.020894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:56.020894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:56.020894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:56.020894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:56.020894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 354/695] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:56.021486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aa68b-fce6-4012-9f29-fa8018736458 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:56.021486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aa68b-fce6-4012-9f29-fa8018736458 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:56.021752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aa68b-fce6-4012-9f29-fa8018736458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:56.045045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-268aa68b-fce6-4012-9f29-fa8018736458 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:56.046712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-268aa68b-fce6-4012-9f29-fa8018736458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:56.120772 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74b742-cb3a-4d66-92c4-5f0e896b7b74 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:56.129730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 355/696] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:56.130842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aa68b-fce6-4012-9f29-fa8018736458 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:56.152317 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:56.161811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041c7a9-3a77-49a5-b3ac-f87f4580d4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:56.162538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041c7a9-3a77-49a5-b3ac-f87f4580d4f3 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:56.162865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041c7a9-3a77-49a5-b3ac-f87f4580d4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:56.163136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041c7a9-3a77-49a5-b3ac-f87f4580d4f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:56.172082 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041c7a9-3a77-49a5-b3ac-f87f4580d4f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:56.173529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041c7a9-3a77-49a5-b3ac-f87f4580d4f3 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-6041c7a9-3a77-49a5-b3ac-f87f4580d4f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:38:56.187030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:56.188068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:56.189030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:56.189030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:56.189030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:38:56.189030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:38:56.189030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:38:56.189030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:38:56.189030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:38:56.189030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:38:56.189030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:38:56.189030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:56.189030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 342/697] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Aug 18 14:38:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:56.194829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041c7a9-3a77-49a5-b3ac-f87f4580d4f3 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:56.241873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:56.313177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041c7a9-3a77-49a5-b3ac-f87f4580d4f3 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:56.335452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aa68b-fce6-4012-9f29-fa8018736458 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:56.364174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3164e19-b1b6-45f6-ad84-8833474d010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:56.365132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3164e19-b1b6-45f6-ad84-8833474d010e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:56.365611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3164e19-b1b6-45f6-ad84-8833474d010e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:56.366081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3164e19-b1b6-45f6-ad84-8833474d010e None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:56.367036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3164e19-b1b6-45f6-ad84-8833474d010e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:56.367541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3164e19-b1b6-45f6-ad84-8833474d010e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:56.371589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3164e19-b1b6-45f6-ad84-8833474d010e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:56.405924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 343/698] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Aug 18 14:38:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:56.407808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc270bf-b8d1-454a-8165-59366ffbd7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:56.408999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc270bf-b8d1-454a-8165-59366ffbd7a9 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:56.409475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc270bf-b8d1-454a-8165-59366ffbd7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:56.409943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc270bf-b8d1-454a-8165-59366ffbd7a9 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:56.410880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc270bf-b8d1-454a-8165-59366ffbd7a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:56.411964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3164e19-b1b6-45f6-ad84-8833474d010e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:56.412872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc270bf-b8d1-454a-8165-59366ffbd7a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:56.414858 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc270bf-b8d1-454a-8165-59366ffbd7a9 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:56.438246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 356/699] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:56.449488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc270bf-b8d1-454a-8165-59366ffbd7a9 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:56.528099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:56.536413 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3164e19-b1b6-45f6-ad84-8833474d010e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:56.578161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:56.597502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80041135-577c-4c17-a50a-06f2b96dce48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:56.598113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80041135-577c-4c17-a50a-06f2b96dce48 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:56.598388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80041135-577c-4c17-a50a-06f2b96dce48 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:56.598762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80041135-577c-4c17-a50a-06f2b96dce48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:56.617943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc270bf-b8d1-454a-8165-59366ffbd7a9 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:56.660044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31102aa6-1b58-4290-a98c-83651de1dbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:56.660872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31102aa6-1b58-4290-a98c-83651de1dbc0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:56.661151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31102aa6-1b58-4290-a98c-83651de1dbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:56.661737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31102aa6-1b58-4290-a98c-83651de1dbc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:56.662729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31102aa6-1b58-4290-a98c-83651de1dbc0 None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=f578727b96564c3db6b5a1e6cac109d8) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:56.663035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31102aa6-1b58-4290-a98c-83651de1dbc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:56.665349 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31102aa6-1b58-4290-a98c-83651de1dbc0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:56.671628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80041135-577c-4c17-a50a-06f2b96dce48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:56.673012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 357/700] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:56 2022] GET /identity/v3/role_assignments?user.id=f578727b96564c3db6b5a1e6cac109d8&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:56.673674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80041135-577c-4c17-a50a-06f2b96dce48 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:56.673799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31102aa6-1b58-4290-a98c-83651de1dbc0 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:56.839882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31102aa6-1b58-4290-a98c-83651de1dbc0 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:56.864211 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:56.872017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 344/701] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:38:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:38:56.873120 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:56.874528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80041135-577c-4c17-a50a-06f2b96dce48 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:56.887153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a0998-462c-40fa-981b-2e33417d9472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:56.887815 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a0998-462c-40fa-981b-2e33417d9472 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:56.888146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a0998-462c-40fa-981b-2e33417d9472 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:56.888417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a0998-462c-40fa-981b-2e33417d9472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:56.900823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a0998-462c-40fa-981b-2e33417d9472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:56.902328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a0998-462c-40fa-981b-2e33417d9472 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-165a0998-462c-40fa-981b-2e33417d9472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:56.917705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:56.918782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:56.919695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:56.919695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:56.919695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:56.919695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 358/702] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:56.924864 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a0998-462c-40fa-981b-2e33417d9472 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:57.007292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80041135-577c-4c17-a50a-06f2b96dce48 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:57.045024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2409e-7dc1-43ce-ad9a-61e2c291646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:57.045656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2409e-7dc1-43ce-ad9a-61e2c291646d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:57.045935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2409e-7dc1-43ce-ad9a-61e2c291646d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:57.046208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2409e-7dc1-43ce-ad9a-61e2c291646d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:57.053609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a0998-462c-40fa-981b-2e33417d9472 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:57.059965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2409e-7dc1-43ce-ad9a-61e2c291646d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:57.061478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2409e-7dc1-43ce-ad9a-61e2c291646d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-3bb2409e-7dc1-43ce-ad9a-61e2c291646d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:38:57.078689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:57.079722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:38:57.080625 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:38:57.080625 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:38:57.080625 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:57.080806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 345/703] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:57.089042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2409e-7dc1-43ce-ad9a-61e2c291646d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:57.097664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd7040-55bc-42ef-8314-73498bbbf915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:57.098273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd7040-55bc-42ef-8314-73498bbbf915 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:57.098552 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd7040-55bc-42ef-8314-73498bbbf915 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:57.098816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd7040-55bc-42ef-8314-73498bbbf915 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:57.099460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd7040-55bc-42ef-8314-73498bbbf915 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:57.099797 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd7040-55bc-42ef-8314-73498bbbf915 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:57.101637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd7040-55bc-42ef-8314-73498bbbf915 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:57.118536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 359/704] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:57.124534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd7040-55bc-42ef-8314-73498bbbf915 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:57.155095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:57.185941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:57.235330 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2409e-7dc1-43ce-ad9a-61e2c291646d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:57.260364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd7040-55bc-42ef-8314-73498bbbf915 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:57.284505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e568a8f-1cc5-46f5-9403-00a5c3a83142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:57.285708 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e568a8f-1cc5-46f5-9403-00a5c3a83142 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:57.285992 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e568a8f-1cc5-46f5-9403-00a5c3a83142 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:57.286494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e568a8f-1cc5-46f5-9403-00a5c3a83142 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:57.287294 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e568a8f-1cc5-46f5-9403-00a5c3a83142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:57.287646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e568a8f-1cc5-46f5-9403-00a5c3a83142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:57.290101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e568a8f-1cc5-46f5-9403-00a5c3a83142 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:57.300651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74b1e1-0c21-452e-a7c8-0db4403049bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:57.301264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74b1e1-0c21-452e-a7c8-0db4403049bd None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:57.301539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74b1e1-0c21-452e-a7c8-0db4403049bd None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:57.301807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74b1e1-0c21-452e-a7c8-0db4403049bd None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:57.302537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74b1e1-0c21-452e-a7c8-0db4403049bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:57.304003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74b1e1-0c21-452e-a7c8-0db4403049bd None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:57.306016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 346/705] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:57.316733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e568a8f-1cc5-46f5-9403-00a5c3a83142 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:57.341852 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 360/706] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:38:57 2022] POST /identity/v3/users => generated 254 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:38:57.438133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a36e1-cc14-475a-82cf-2dee21c7b065 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:57.438990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e568a8f-1cc5-46f5-9403-00a5c3a83142 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:57.439395 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a36e1-cc14-475a-82cf-2dee21c7b065 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:57.439949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a36e1-cc14-475a-82cf-2dee21c7b065 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:57.441936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 361/707] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:57.448139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3777e9-eb41-4494-9526-510cb2e98207 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:57.448778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3777e9-eb41-4494-9526-510cb2e98207 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:57.449261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3777e9-eb41-4494-9526-510cb2e98207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:57.450077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:57.488181 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-3c3777e9-eb41-4494-9526-510cb2e98207 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:57.489931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3c3777e9-eb41-4494-9526-510cb2e98207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:57.501125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d62a87-7487-447f-b5d3-2db4ddb54859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:57.501783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d62a87-7487-447f-b5d3-2db4ddb54859 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:57.502061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d62a87-7487-447f-b5d3-2db4ddb54859 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:57.502332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d62a87-7487-447f-b5d3-2db4ddb54859 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:57.517576 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d62a87-7487-447f-b5d3-2db4ddb54859 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:57.519088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d62a87-7487-447f-b5d3-2db4ddb54859 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-60d62a87-7487-447f-b5d3-2db4ddb54859 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:38:57.544899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:57.545888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:57.546847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:38:57.546847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:38:57.546847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:38:57.546847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:38:57.546847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:38:57.546847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:38:57.546847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:38:57.546847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:57.550031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 347/708] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:57.556680 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d62a87-7487-447f-b5d3-2db4ddb54859 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:57.593097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 362/709] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:57.669658 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d62a87-7487-447f-b5d3-2db4ddb54859 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:57.707405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617e402-f233-49e5-af49-09f04f006dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:57.708268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617e402-f233-49e5-af49-09f04f006dd0 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:57.708698 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617e402-f233-49e5-af49-09f04f006dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:57.709140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617e402-f233-49e5-af49-09f04f006dd0 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:57.709977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617e402-f233-49e5-af49-09f04f006dd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:57.710446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617e402-f233-49e5-af49-09f04f006dd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:57.712065 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617e402-f233-49e5-af49-09f04f006dd0 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:57.727333 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 348/710] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:38:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:57.733207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3777e9-eb41-4494-9526-510cb2e98207 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:57.766258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:57.858810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af85183-a44d-4c99-9ea4-f36d38942928 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:57.859363 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af85183-a44d-4c99-9ea4-f36d38942928 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:57.859916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af85183-a44d-4c99-9ea4-f36d38942928 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:57.861451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 349/711] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:57.866207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3777e9-eb41-4494-9526-510cb2e98207 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:57.870562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fe59e-86c4-4cfe-9274-1811be46e018 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:57.870850 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fe59e-86c4-4cfe-9274-1811be46e018 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:57.871121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fe59e-86c4-4cfe-9274-1811be46e018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:57.907705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4b7fe59e-86c4-4cfe-9274-1811be46e018 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:57.907705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4b7fe59e-86c4-4cfe-9274-1811be46e018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:57.920045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9ba5a-eb67-4ed9-8bdc-e4db9787d167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:57.920671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ba5a-eb67-4ed9-8bdc-e4db9787d167 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:57.920950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ba5a-eb67-4ed9-8bdc-e4db9787d167 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:57.921225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ba5a-eb67-4ed9-8bdc-e4db9787d167 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:57.922008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9ba5a-eb67-4ed9-8bdc-e4db9787d167 None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=3f3efa9f3eb24864943b68d241435817) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:57.922317 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9ba5a-eb67-4ed9-8bdc-e4db9787d167 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:57.923769 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9ba5a-eb67-4ed9-8bdc-e4db9787d167 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:57.935385 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 363/712] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:38:57 2022] GET /identity/v3/role_assignments?user.id=3f3efa9f3eb24864943b68d241435817&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=9a439cf8eef34ada85428b378878e63e => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:57.948294 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:58.002014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9ba5a-eb67-4ed9-8bdc-e4db9787d167 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:58.020614 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 350/713] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:58.110129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:58.118146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9ba5a-eb67-4ed9-8bdc-e4db9787d167 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:58.165411 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf7c84-fe34-4bf5-81cb-8f547087720b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:58.166443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf7c84-fe34-4bf5-81cb-8f547087720b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:58.167157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf7c84-fe34-4bf5-81cb-8f547087720b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:58.167673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf7c84-fe34-4bf5-81cb-8f547087720b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:58.168571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf7c84-fe34-4bf5-81cb-8f547087720b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:58.170182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf7c84-fe34-4bf5-81cb-8f547087720b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-01bf7c84-fe34-4bf5-81cb-8f547087720b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:58.190853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:58.192049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:38:58.192898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 364/714] 10.222.0.36 () {56 vars in 1129 bytes} [Thu Aug 18 14:38:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:38:58.197730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7fe59e-86c4-4cfe-9274-1811be46e018 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:58.307486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7fe59e-86c4-4cfe-9274-1811be46e018 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:58.351360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990b549-43c9-4e54-85ef-594894147d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:58.352400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990b549-43c9-4e54-85ef-594894147d63 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:58.352867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990b549-43c9-4e54-85ef-594894147d63 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:58.353320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990b549-43c9-4e54-85ef-594894147d63 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:58.354152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990b549-43c9-4e54-85ef-594894147d63 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:58.354643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990b549-43c9-4e54-85ef-594894147d63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:58.356342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990b549-43c9-4e54-85ef-594894147d63 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:58.363035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 351/715] 10.222.0.36 () {56 vars in 1124 bytes} [Thu Aug 18 14:38:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:58.368534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf7c84-fe34-4bf5-81cb-8f547087720b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:58.410016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990b549-43c9-4e54-85ef-594894147d63 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:58.436394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:58.492440 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf7c84-fe34-4bf5-81cb-8f547087720b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:58.519800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990b549-43c9-4e54-85ef-594894147d63 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:58.527207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14982d-03a7-4f01-8c16-afa1be481142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:58.527844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14982d-03a7-4f01-8c16-afa1be481142 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:58.528120 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14982d-03a7-4f01-8c16-afa1be481142 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:58.528389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14982d-03a7-4f01-8c16-afa1be481142 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:58.529211 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14982d-03a7-4f01-8c16-afa1be481142 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:58.529338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14982d-03a7-4f01-8c16-afa1be481142 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:58.530643 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14982d-03a7-4f01-8c16-afa1be481142 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:58.536795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 365/716] 10.222.0.36 () {56 vars in 1124 bytes} [Thu Aug 18 14:38:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:58.572260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25594222-d0ef-4a03-9c9d-57d3374b043d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:58.572855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25594222-d0ef-4a03-9c9d-57d3374b043d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:58.573141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25594222-d0ef-4a03-9c9d-57d3374b043d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:58.573416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25594222-d0ef-4a03-9c9d-57d3374b043d None admin] PATH_INFO: `/v3/services/network` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:58.574025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25594222-d0ef-4a03-9c9d-57d3374b043d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:58.575426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25594222-d0ef-4a03-9c9d-57d3374b043d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-25594222-d0ef-4a03-9c9d-57d3374b043d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:38:58.599719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:38:58.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 14:38:58.601642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:38:58.604730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 352/717] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:38:58 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:38:58.610212 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14982d-03a7-4f01-8c16-afa1be481142 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:58.660903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:58.746568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14982d-03a7-4f01-8c16-afa1be481142 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:58.782357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e3745-5883-4d27-8dee-e945ac57d918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:58.782972 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e3745-5883-4d27-8dee-e945ac57d918 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:58.783239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e3745-5883-4d27-8dee-e945ac57d918 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:58.783508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e3745-5883-4d27-8dee-e945ac57d918 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:58.784165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e3745-5883-4d27-8dee-e945ac57d918 None admin] RBAC: Adding query filter params (name=network) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:58.784467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e3745-5883-4d27-8dee-e945ac57d918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:58.785818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e3745-5883-4d27-8dee-e945ac57d918 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:58.790649 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 366/718] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:38:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:58.796230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25594222-d0ef-4a03-9c9d-57d3374b043d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:58.908067 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:58.921329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25594222-d0ef-4a03-9c9d-57d3374b043d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:38:58.954604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f4f76-2515-4d71-8291-940cc677e98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:38:58.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f4f76-2515-4d71-8291-940cc677e98f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:58.955552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f4f76-2515-4d71-8291-940cc677e98f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:58.955853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f4f76-2515-4d71-8291-940cc677e98f None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:58.956486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f4f76-2515-4d71-8291-940cc677e98f None admin] RBAC: Adding query filter params (type=network) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:38:58.956784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f4f76-2515-4d71-8291-940cc677e98f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:38:58.958170 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f4f76-2515-4d71-8291-940cc677e98f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:38:58.964443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 353/719] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:38:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:38:59.232929 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e76984-d5fe-4162-bd42-e89f7c5b2bf5 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:59.233266 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e76984-d5fe-4162-bd42-e89f7c5b2bf5 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:59.233485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e76984-d5fe-4162-bd42-e89f7c5b2bf5 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:59.235184 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 367/720] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:38:59.240403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d718b0-0c5e-4086-9ad0-be32fbce32a2 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:59.240846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d718b0-0c5e-4086-9ad0-be32fbce32a2 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:59.241014 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d718b0-0c5e-4086-9ad0-be32fbce32a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:59.260601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-49d718b0-0c5e-4086-9ad0-be32fbce32a2 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:59.261800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-49d718b0-0c5e-4086-9ad0-be32fbce32a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:59.357951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 354/721] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:38:59.779458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e76984-d5fe-4162-bd42-e89f7c5b2bf5 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:38:59.815739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad756e-6d42-4e78-b439-7cebd40f7020 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:59.816406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad756e-6d42-4e78-b439-7cebd40f7020 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:59.816912 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad756e-6d42-4e78-b439-7cebd40f7020 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:59.818720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 355/722] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:38:59.823841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32a252-7fbf-4ad7-b7bd-528fba912803 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:38:59.824383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32a252-7fbf-4ad7-b7bd-528fba912803 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:38:59.824876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32a252-7fbf-4ad7-b7bd-528fba912803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:38:59.845740 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-2e32a252-7fbf-4ad7-b7bd-528fba912803 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:38:59.848117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2e32a252-7fbf-4ad7-b7bd-528fba912803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:38:59.928415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:38:59.969273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 356/723] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 14:38:59.977026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e76984-d5fe-4162-bd42-e89f7c5b2bf5 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:00.021172 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd19f1b-db28-4b8d-bd0c-abf1838bb860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:00.021803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd19f1b-db28-4b8d-bd0c-abf1838bb860 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:00.022083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd19f1b-db28-4b8d-bd0c-abf1838bb860 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:00.022359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd19f1b-db28-4b8d-bd0c-abf1838bb860 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:00.032956 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd19f1b-db28-4b8d-bd0c-abf1838bb860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-cbd19f1b-db28-4b8d-bd0c-abf1838bb860 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:00.037617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:00.038777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:00.040047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 368/724] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:38:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:00.044609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32a252-7fbf-4ad7-b7bd-528fba912803 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:00.060017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:00.149331 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32a252-7fbf-4ad7-b7bd-528fba912803 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:00.190496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b8c82-8d59-425d-b5c8-44e44e9e0a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:00.191101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b8c82-8d59-425d-b5c8-44e44e9e0a32 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:00.191383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b8c82-8d59-425d-b5c8-44e44e9e0a32 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:00.191659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b8c82-8d59-425d-b5c8-44e44e9e0a32 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:00.192402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b8c82-8d59-425d-b5c8-44e44e9e0a32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:00.192711 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b8c82-8d59-425d-b5c8-44e44e9e0a32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:00.194082 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b8c82-8d59-425d-b5c8-44e44e9e0a32 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:00.204854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 357/725] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:39:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:00.210149 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd19f1b-db28-4b8d-bd0c-abf1838bb860 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:00.329609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd19f1b-db28-4b8d-bd0c-abf1838bb860 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:00.333482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b8c82-8d59-425d-b5c8-44e44e9e0a32 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:00.369018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eafefef-0d9a-43b5-aaf4-b06df90ae071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:00.369926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafefef-0d9a-43b5-aaf4-b06df90ae071 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:00.370396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafefef-0d9a-43b5-aaf4-b06df90ae071 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:00.370837 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafefef-0d9a-43b5-aaf4-b06df90ae071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:00.385726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eafefef-0d9a-43b5-aaf4-b06df90ae071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:00.387519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eafefef-0d9a-43b5-aaf4-b06df90ae071 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-2eafefef-0d9a-43b5-aaf4-b06df90ae071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:00.402729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:00.403949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:00.404884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:00.404884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:00.404884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:00.407531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 369/726] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:00.411980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:00.416725 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eafefef-0d9a-43b5-aaf4-b06df90ae071 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:00.444050 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:00.522507 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b8c82-8d59-425d-b5c8-44e44e9e0a32 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:00.539772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eafefef-0d9a-43b5-aaf4-b06df90ae071 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:00.563028 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f8850-317e-400b-bc44-80736450ac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:00.563655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f8850-317e-400b-bc44-80736450ac01 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:00.564070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f8850-317e-400b-bc44-80736450ac01 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:00.564342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f8850-317e-400b-bc44-80736450ac01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:00.575359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f8850-317e-400b-bc44-80736450ac01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:00.576380 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a28e5-f875-43b6-9128-6152aeacd761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:00.576987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a28e5-f875-43b6-9128-6152aeacd761 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:00.577260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a28e5-f875-43b6-9128-6152aeacd761 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:00.577525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a28e5-f875-43b6-9128-6152aeacd761 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:00.578763 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a28e5-f875-43b6-9128-6152aeacd761 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:00.579066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a28e5-f875-43b6-9128-6152aeacd761 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:00.580515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a28e5-f875-43b6-9128-6152aeacd761 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-bd2f8850-317e-400b-bc44-80736450ac01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:00.581400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:00.582312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:00.583150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 358/727] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:39:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:00.584357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0c8f7-3dca-46f7-8ab5-c4f2b7111def None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:00.584525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0c8f7-3dca-46f7-8ab5-c4f2b7111def None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:00.587161 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0c8f7-3dca-46f7-8ab5-c4f2b7111def None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:00.587161 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 359/728] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:00.591361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b8489-d341-4a44-a760-2f5a38057147 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:00.591638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b8489-d341-4a44-a760-2f5a38057147 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:00.591942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b8489-d341-4a44-a760-2f5a38057147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:00.603342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 370/729] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:00.604438 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a28e5-f875-43b6-9128-6152aeacd761 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:00.611753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-896b8489-d341-4a44-a760-2f5a38057147 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:00.612964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-896b8489-d341-4a44-a760-2f5a38057147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:00.670999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:00.683275 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 360/730] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:39:00.688500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b8489-d341-4a44-a760-2f5a38057147 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:00.719385 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a28e5-f875-43b6-9128-6152aeacd761 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:00.760604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df3990-a05c-4b18-a63e-4c1d16a030bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:00.761221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df3990-a05c-4b18-a63e-4c1d16a030bc None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:00.761496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df3990-a05c-4b18-a63e-4c1d16a030bc None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:00.761770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df3990-a05c-4b18-a63e-4c1d16a030bc None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:00.762429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df3990-a05c-4b18-a63e-4c1d16a030bc None admin] RBAC: Adding query filter params (name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:00.762729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df3990-a05c-4b18-a63e-4c1d16a030bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:00.764132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df3990-a05c-4b18-a63e-4c1d16a030bc None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:00.780056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 371/731] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:39:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:00.787949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df3990-a05c-4b18-a63e-4c1d16a030bc None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:00.802367 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:00.876410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b8489-d341-4a44-a760-2f5a38057147 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:00.950516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44ab04-cdb4-4997-8c7a-9da545c82e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:00.950885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df3990-a05c-4b18-a63e-4c1d16a030bc None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:00.951159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44ab04-cdb4-4997-8c7a-9da545c82e9f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:00.951515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44ab04-cdb4-4997-8c7a-9da545c82e9f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:00.951813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44ab04-cdb4-4997-8c7a-9da545c82e9f None admin] PATH_INFO: `/v3/users/swift` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:00.968754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44ab04-cdb4-4997-8c7a-9da545c82e9f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:00.970213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44ab04-cdb4-4997-8c7a-9da545c82e9f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:00.974300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-9e44ab04-cdb4-4997-8c7a-9da545c82e9f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:00.982709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:00.984002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:00.989538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:00.989538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:00.989538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:00.989538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:00.989538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:00.989538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:00.989538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:00.989538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:00.989538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 14:39:00.989538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:00.989538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 361/732] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:39:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:00.992018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44ab04-cdb4-4997-8c7a-9da545c82e9f None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:00.999030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb27041-8be2-4d86-85b1-80598e339f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:01.000466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb27041-8be2-4d86-85b1-80598e339f69 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.000987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb27041-8be2-4d86-85b1-80598e339f69 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.001462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb27041-8be2-4d86-85b1-80598e339f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.012763 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb27041-8be2-4d86-85b1-80598e339f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:01.015950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb27041-8be2-4d86-85b1-80598e339f69 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-feb27041-8be2-4d86-85b1-80598e339f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:01.030981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:01.032044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:01.032946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:01.032946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:01.032946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:01.039221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 372/733] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:01.040725 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85661-ab92-48f8-b83d-00008d42f4df None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.041188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85661-ab92-48f8-b83d-00008d42f4df None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.041662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85661-ab92-48f8-b83d-00008d42f4df None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.043739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 373/734] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:01.044770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c85661-ab92-48f8-b83d-00008d42f4df None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:01.125648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44ab04-cdb4-4997-8c7a-9da545c82e9f None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:01.150073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:01.153362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c85661-ab92-48f8-b83d-00008d42f4df None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:01.162774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd285a3f-a4e1-40db-9c63-65da15e55d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:01.163395 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd285a3f-a4e1-40db-9c63-65da15e55d2a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.163669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd285a3f-a4e1-40db-9c63-65da15e55d2a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.163967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd285a3f-a4e1-40db-9c63-65da15e55d2a None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.164702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd285a3f-a4e1-40db-9c63-65da15e55d2a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:01.165011 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd285a3f-a4e1-40db-9c63-65da15e55d2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:01.166449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd285a3f-a4e1-40db-9c63-65da15e55d2a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:01.190046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2106f-b515-48d7-b405-9534270ad4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:01.190718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 362/735] 10.222.0.36 () {56 vars in 1135 bytes} [Thu Aug 18 14:39:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:01.190873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2106f-b515-48d7-b405-9534270ad4b3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.190971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2106f-b515-48d7-b405-9534270ad4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.191178 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2106f-b515-48d7-b405-9534270ad4b3 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.191844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2106f-b515-48d7-b405-9534270ad4b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:01.191947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a8fd5-44fc-4a44-aa29-623d0541194d None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.192149 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2106f-b515-48d7-b405-9534270ad4b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:01.192248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a8fd5-44fc-4a44-aa29-623d0541194d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.192464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a8fd5-44fc-4a44-aa29-623d0541194d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.193524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2106f-b515-48d7-b405-9534270ad4b3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:01.206199 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 374/736] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:01.206845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2106f-b515-48d7-b405-9534270ad4b3 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:01.215966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-031a8fd5-44fc-4a44-aa29-623d0541194d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:01.217155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-031a8fd5-44fc-4a44-aa29-623d0541194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:01.239101 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:01.300051 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 363/737] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:39:01.300974 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a8fd5-44fc-4a44-aa29-623d0541194d None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:01.315716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2106f-b515-48d7-b405-9534270ad4b3 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:01.351902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433561e-2791-46d6-9e7e-b8a5ad213452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:01.352774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433561e-2791-46d6-9e7e-b8a5ad213452 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.353295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433561e-2791-46d6-9e7e-b8a5ad213452 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.353716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433561e-2791-46d6-9e7e-b8a5ad213452 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.365474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433561e-2791-46d6-9e7e-b8a5ad213452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-7433561e-2791-46d6-9e7e-b8a5ad213452 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:01.371102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:01.372250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:01.373258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:01.373258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:01.373258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:01.373258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:01.373258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:01.373258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:01.373258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:01.373258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:01.373258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:01.373258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:01.373258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:01.373258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:01.382661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 375/738] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:39:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:01.383719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433561e-2791-46d6-9e7e-b8a5ad213452 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:01.418190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a8fd5-44fc-4a44-aa29-623d0541194d None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:01.430163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:01.456665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb45f36-e24b-4885-8ca8-76904c4000f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:01.457277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb45f36-e24b-4885-8ca8-76904c4000f9 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.457553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb45f36-e24b-4885-8ca8-76904c4000f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.457824 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb45f36-e24b-4885-8ca8-76904c4000f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.481466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:01.491633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433561e-2791-46d6-9e7e-b8a5ad213452 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:01.525546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35c99b-36fe-4385-96e9-f9852d404af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:01.526170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35c99b-36fe-4385-96e9-f9852d404af6 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.526451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35c99b-36fe-4385-96e9-f9852d404af6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.526723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35c99b-36fe-4385-96e9-f9852d404af6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.534142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35c99b-36fe-4385-96e9-f9852d404af6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:01.534490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb45f36-e24b-4885-8ca8-76904c4000f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:01.535573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35c99b-36fe-4385-96e9-f9852d404af6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:01.536410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb45f36-e24b-4885-8ca8-76904c4000f9 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-dd35c99b-36fe-4385-96e9-f9852d404af6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:01.547273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:01.548262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:01.549215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:01.549215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:01.549215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:01.549215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:01.549215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:01.549215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:01.549215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:01.549215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:01.549215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 14:39:01.549215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:01.551982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 376/739] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Aug 18 14:39:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:01.553191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35c99b-36fe-4385-96e9-f9852d404af6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:01.638015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 364/740] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:39:01.639726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749d34f-de38-4fc8-9cb2-67fe4851536f None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.640020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749d34f-de38-4fc8-9cb2-67fe4851536f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.640321 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749d34f-de38-4fc8-9cb2-67fe4851536f None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.641594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 365/741] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:01.642604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749d34f-de38-4fc8-9cb2-67fe4851536f None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:01.655073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:01.655458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35c99b-36fe-4385-96e9-f9852d404af6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:01.689219 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd43451-ac79-4419-b7fa-8d31c3ff39ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:01.689817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd43451-ac79-4419-b7fa-8d31c3ff39ea None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.690094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd43451-ac79-4419-b7fa-8d31c3ff39ea None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.690363 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd43451-ac79-4419-b7fa-8d31c3ff39ea None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.691018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd43451-ac79-4419-b7fa-8d31c3ff39ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:01.691315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd43451-ac79-4419-b7fa-8d31c3ff39ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:01.692718 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd43451-ac79-4419-b7fa-8d31c3ff39ea None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:01.700667 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 377/742] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:39:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:01.701433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd43451-ac79-4419-b7fa-8d31c3ff39ea None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:01.715942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:01.750494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749d34f-de38-4fc8-9cb2-67fe4851536f None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:01.783617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09300ffe-5a67-44f7-bf20-f661f78978d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:01.784239 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09300ffe-5a67-44f7-bf20-f661f78978d4 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.784517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09300ffe-5a67-44f7-bf20-f661f78978d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.784814 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09300ffe-5a67-44f7-bf20-f661f78978d4 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.785534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09300ffe-5a67-44f7-bf20-f661f78978d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:01.785838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09300ffe-5a67-44f7-bf20-f661f78978d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:01.787258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09300ffe-5a67-44f7-bf20-f661f78978d4 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:01.798016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd43451-ac79-4419-b7fa-8d31c3ff39ea None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:01.804516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 366/743] 10.222.0.36 () {56 vars in 1149 bytes} [Thu Aug 18 14:39:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:01.805230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09300ffe-5a67-44f7-bf20-f661f78978d4 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:01.830249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74729d62-be04-47da-9da5-88255a1d7454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:01.830849 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74729d62-be04-47da-9da5-88255a1d7454 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.831126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74729d62-be04-47da-9da5-88255a1d7454 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.831397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74729d62-be04-47da-9da5-88255a1d7454 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.832061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74729d62-be04-47da-9da5-88255a1d7454 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:01.833412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74729d62-be04-47da-9da5-88255a1d7454 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:01.849345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 378/744] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:39:01 2022] POST /identity/v3/services => generated 251 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:01.850783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2d9df-817b-40d7-b080-e1fe3c5a0ed5 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.851070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2d9df-817b-40d7-b080-e1fe3c5a0ed5 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.851345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2d9df-817b-40d7-b080-e1fe3c5a0ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.873605 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:01.878535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-2cb2d9df-817b-40d7-b080-e1fe3c5a0ed5 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:01.880797 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2cb2d9df-817b-40d7-b080-e1fe3c5a0ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:01.915668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09300ffe-5a67-44f7-bf20-f661f78978d4 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:01.966863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a28266-e7a3-4147-8f11-f080dd7b9d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:01.967510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a28266-e7a3-4147-8f11-f080dd7b9d26 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:01.967885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a28266-e7a3-4147-8f11-f080dd7b9d26 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:01.968228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a28266-e7a3-4147-8f11-f080dd7b9d26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:01.973915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:01.980178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a28266-e7a3-4147-8f11-f080dd7b9d26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:01.981118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 379/745] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 14:39:01.982174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2d9df-817b-40d7-b080-e1fe3c5a0ed5 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:01.993133 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a28266-e7a3-4147-8f11-f080dd7b9d26 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-f6a28266-e7a3-4147-8f11-f080dd7b9d26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:02.016033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:02.017188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:02.018354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:02.018354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:02.018354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:02.022160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 367/746] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:02.022609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a28266-e7a3-4147-8f11-f080dd7b9d26 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:02.108614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2d9df-817b-40d7-b080-e1fe3c5a0ed5 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:02.144661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b3827-e9d7-425e-abd9-cdba4004ccac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:02.145297 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b3827-e9d7-425e-abd9-cdba4004ccac None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:02.145581 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b3827-e9d7-425e-abd9-cdba4004ccac None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:02.145854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b3827-e9d7-425e-abd9-cdba4004ccac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:02.146218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a28266-e7a3-4147-8f11-f080dd7b9d26 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:02.156782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b3827-e9d7-425e-abd9-cdba4004ccac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:02.158380 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b3827-e9d7-425e-abd9-cdba4004ccac None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:02.163522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-496b3827-e9d7-425e-abd9-cdba4004ccac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:02.176526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:02.177505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:02.178412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:02.178412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:02.178412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:02.178412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 380/747] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:02.179054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b3827-e9d7-425e-abd9-cdba4004ccac None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:02.188313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4f4b7-d870-445e-9677-db0e70f01978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:02.189035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4f4b7-d870-445e-9677-db0e70f01978 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:02.189318 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4f4b7-d870-445e-9677-db0e70f01978 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:02.189587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4f4b7-d870-445e-9677-db0e70f01978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:02.235864 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:02.264664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4f4b7-d870-445e-9677-db0e70f01978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:02.266528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4f4b7-d870-445e-9677-db0e70f01978 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:02.288868 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b3827-e9d7-425e-abd9-cdba4004ccac None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:02.326298 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5fcc0-9aff-4e13-929a-c9c26f067b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:02.327009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5fcc0-9aff-4e13-929a-c9c26f067b1b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:02.327406 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5fcc0-9aff-4e13-929a-c9c26f067b1b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:02.327736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5fcc0-9aff-4e13-929a-c9c26f067b1b None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:02.328437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5fcc0-9aff-4e13-929a-c9c26f067b1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:02.328742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5fcc0-9aff-4e13-929a-c9c26f067b1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:02.330111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5fcc0-9aff-4e13-929a-c9c26f067b1b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:02.348118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 381/748] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:02.348118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5fcc0-9aff-4e13-929a-c9c26f067b1b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:02.372582 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 368/749] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 14:39:02.373485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4f4b7-d870-445e-9677-db0e70f01978 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:02.395168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:02.456721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5fcc0-9aff-4e13-929a-c9c26f067b1b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:02.470552 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:02.492757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab98776-0966-4659-ad6f-051fe018eafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:02.493304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab98776-0966-4659-ad6f-051fe018eafd None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:02.493580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab98776-0966-4659-ad6f-051fe018eafd None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:02.493848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab98776-0966-4659-ad6f-051fe018eafd None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:02.494492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab98776-0966-4659-ad6f-051fe018eafd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:02.494793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab98776-0966-4659-ad6f-051fe018eafd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:02.496208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab98776-0966-4659-ad6f-051fe018eafd None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:02.496490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4f4b7-d870-445e-9677-db0e70f01978 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:02.507806 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 382/750] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:02.508597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab98776-0966-4659-ad6f-051fe018eafd None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:02.540297 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0020636-ec12-4662-a78f-816a30887565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:02.541277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0020636-ec12-4662-a78f-816a30887565 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:02.541709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0020636-ec12-4662-a78f-816a30887565 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:02.542117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0020636-ec12-4662-a78f-816a30887565 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:02.543024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0020636-ec12-4662-a78f-816a30887565 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:02.544677 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0020636-ec12-4662-a78f-816a30887565 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:02.561433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 369/751] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:39:02 2022] POST /identity/v3/services => generated 240 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:02.562630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0020636-ec12-4662-a78f-816a30887565 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:02.633500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab98776-0966-4659-ad6f-051fe018eafd None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:02.677806 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0f99-cbf6-496d-bb20-981c3025005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:02.678897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0f99-cbf6-496d-bb20-981c3025005b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:02.679361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0f99-cbf6-496d-bb20-981c3025005b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:02.679910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0f99-cbf6-496d-bb20-981c3025005b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:02.681209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:02.691186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c0f99-cbf6-496d-bb20-981c3025005b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:02.693264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c0f99-cbf6-496d-bb20-981c3025005b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-143c0f99-cbf6-496d-bb20-981c3025005b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:02.709201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:02.710148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:02.711030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:02.711030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:02.711030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:02.711030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:02.711030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:02.711030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:02.711030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:02.711030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:02.711475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0020636-ec12-4662-a78f-816a30887565 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:02.711558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 383/752] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:02.711558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0f99-cbf6-496d-bb20-981c3025005b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:02.760187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4488975-0d0f-4eb6-a4af-841935f9c45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:02.760766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4488975-0d0f-4eb6-a4af-841935f9c45c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:02.761045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4488975-0d0f-4eb6-a4af-841935f9c45c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:02.761318 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4488975-0d0f-4eb6-a4af-841935f9c45c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:02.776577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4488975-0d0f-4eb6-a4af-841935f9c45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:02.778083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4488975-0d0f-4eb6-a4af-841935f9c45c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-f4488975-0d0f-4eb6-a4af-841935f9c45c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:02.796689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:02.797723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:02.798621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:02.798621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:02.798621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:02.800905 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 370/753] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:02.802011 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4488975-0d0f-4eb6-a4af-841935f9c45c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:02.859975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:02.929395 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4488975-0d0f-4eb6-a4af-841935f9c45c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:02.963560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5113f63-cdb8-4b76-b9bf-6c3f64512243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:02.964200 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5113f63-cdb8-4b76-b9bf-6c3f64512243 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:02.964480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5113f63-cdb8-4b76-b9bf-6c3f64512243 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:02.964752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5113f63-cdb8-4b76-b9bf-6c3f64512243 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:02.965061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0f99-cbf6-496d-bb20-981c3025005b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:02.965446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5113f63-cdb8-4b76-b9bf-6c3f64512243 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:02.965751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5113f63-cdb8-4b76-b9bf-6c3f64512243 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:02.967187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5113f63-cdb8-4b76-b9bf-6c3f64512243 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:02.984635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:02.985839 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 371/754] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:02.986729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5113f63-cdb8-4b76-b9bf-6c3f64512243 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:03.007797 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46272e7d-bc10-40b2-afbe-425cc12a88d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:03.008445 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46272e7d-bc10-40b2-afbe-425cc12a88d9 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:03.008721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46272e7d-bc10-40b2-afbe-425cc12a88d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:03.008998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46272e7d-bc10-40b2-afbe-425cc12a88d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:03.017327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46272e7d-bc10-40b2-afbe-425cc12a88d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:03.018687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46272e7d-bc10-40b2-afbe-425cc12a88d9 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-46272e7d-bc10-40b2-afbe-425cc12a88d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:03.030437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:03.031444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:03.032542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:03.032542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:03.032542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:03.032542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:03.032542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:03.032542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:03.032542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:03.032542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:03.032542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:39:03.032542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:03.032542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 384/755] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Aug 18 14:39:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:03.033157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46272e7d-bc10-40b2-afbe-425cc12a88d9 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:03.124216 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5113f63-cdb8-4b76-b9bf-6c3f64512243 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:03.130732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:03.146769 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46272e7d-bc10-40b2-afbe-425cc12a88d9 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:03.164807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72df3b8-4e55-41a8-898c-280d8249e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:03.165433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72df3b8-4e55-41a8-898c-280d8249e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:03.165715 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72df3b8-4e55-41a8-898c-280d8249e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:03.165989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72df3b8-4e55-41a8-898c-280d8249e0f2 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:03.166904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72df3b8-4e55-41a8-898c-280d8249e0f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:03.167209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72df3b8-4e55-41a8-898c-280d8249e0f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:03.168776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72df3b8-4e55-41a8-898c-280d8249e0f2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:03.188854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 372/756] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:03.188854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72df3b8-4e55-41a8-898c-280d8249e0f2 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:03.197265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e105f-6233-45eb-b14c-0841d1c7fe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:03.198519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e105f-6233-45eb-b14c-0841d1c7fe59 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:03.199013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e105f-6233-45eb-b14c-0841d1c7fe59 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:03.199486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e105f-6233-45eb-b14c-0841d1c7fe59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:03.201004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e105f-6233-45eb-b14c-0841d1c7fe59 None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=f578727b96564c3db6b5a1e6cac109d8) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:03.201497 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e105f-6233-45eb-b14c-0841d1c7fe59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:03.203745 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e105f-6233-45eb-b14c-0841d1c7fe59 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:03.210883 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 385/757] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:39:03 2022] GET /identity/v3/role_assignments?user.id=f578727b96564c3db6b5a1e6cac109d8&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=9a439cf8eef34ada85428b378878e63e => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:03.211873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e105f-6233-45eb-b14c-0841d1c7fe59 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:03.267412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:03.326941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72df3b8-4e55-41a8-898c-280d8249e0f2 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:03.354658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e105f-6233-45eb-b14c-0841d1c7fe59 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:03.365168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d614808-e022-467d-9d5b-c01d4f390d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:03.365781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d614808-e022-467d-9d5b-c01d4f390d51 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:03.366062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d614808-e022-467d-9d5b-c01d4f390d51 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:03.366334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d614808-e022-467d-9d5b-c01d4f390d51 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:03.367071 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d614808-e022-467d-9d5b-c01d4f390d51 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:03.367380 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d614808-e022-467d-9d5b-c01d4f390d51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:03.368843 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d614808-e022-467d-9d5b-c01d4f390d51 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:03.400658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd196a1-1196-4aa2-9fef-b1ef565cce23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:03.402756 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd196a1-1196-4aa2-9fef-b1ef565cce23 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:03.403285 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd196a1-1196-4aa2-9fef-b1ef565cce23 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:03.403850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd196a1-1196-4aa2-9fef-b1ef565cce23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:03.412215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd196a1-1196-4aa2-9fef-b1ef565cce23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:03.414250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd196a1-1196-4aa2-9fef-b1ef565cce23 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:03.417668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 373/758] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Aug 18 14:39:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:03.422530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d614808-e022-467d-9d5b-c01d4f390d51 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-6dd196a1-1196-4aa2-9fef-b1ef565cce23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:03.428526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:03.429474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:03.430364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:03.430364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:03.430364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:03.430364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:03.430364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:03.430364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:03.430364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:03.430364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:03.433037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 386/759] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:03.440418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd196a1-1196-4aa2-9fef-b1ef565cce23 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:03.458627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:03.565505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd196a1-1196-4aa2-9fef-b1ef565cce23 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:03.603171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cee4a-b9e0-4dbe-a5cb-a7552428efc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:03.604204 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cee4a-b9e0-4dbe-a5cb-a7552428efc5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:03.604698 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cee4a-b9e0-4dbe-a5cb-a7552428efc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:03.605184 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cee4a-b9e0-4dbe-a5cb-a7552428efc5 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:03.606069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cee4a-b9e0-4dbe-a5cb-a7552428efc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:03.606597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cee4a-b9e0-4dbe-a5cb-a7552428efc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:03.608261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cee4a-b9e0-4dbe-a5cb-a7552428efc5 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:03.623213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 387/760] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:03.629820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d614808-e022-467d-9d5b-c01d4f390d51 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:03.630693 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cee4a-b9e0-4dbe-a5cb-a7552428efc5 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:03.644208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:03.674695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f46085-3bce-4832-bf9a-23e2da11e664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:03.675323 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f46085-3bce-4832-bf9a-23e2da11e664 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:03.675603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f46085-3bce-4832-bf9a-23e2da11e664 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:03.675958 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f46085-3bce-4832-bf9a-23e2da11e664 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:03.686880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f46085-3bce-4832-bf9a-23e2da11e664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:03.688504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f46085-3bce-4832-bf9a-23e2da11e664 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-97f46085-3bce-4832-bf9a-23e2da11e664 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:03.704386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:03.705357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:03.706259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:03.706259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:03.706259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:03.706259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 374/761] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:03.708361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:03.711387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f46085-3bce-4832-bf9a-23e2da11e664 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:03.742674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cee4a-b9e0-4dbe-a5cb-a7552428efc5 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:03.779395 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda1196-79fe-4010-bf5e-5b91f9cf1af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:03.780382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda1196-79fe-4010-bf5e-5b91f9cf1af5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:03.780927 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda1196-79fe-4010-bf5e-5b91f9cf1af5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:03.781417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda1196-79fe-4010-bf5e-5b91f9cf1af5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:03.782489 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda1196-79fe-4010-bf5e-5b91f9cf1af5 None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=cbb13837e1694d7aa928ac855e21d1a9) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:03.783022 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda1196-79fe-4010-bf5e-5b91f9cf1af5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:03.784721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda1196-79fe-4010-bf5e-5b91f9cf1af5 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:03.791075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 388/762] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:39:03 2022] GET /identity/v3/role_assignments?user.id=cbb13837e1694d7aa928ac855e21d1a9&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:03.821465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f46085-3bce-4832-bf9a-23e2da11e664 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:03.860039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef683b-e782-40b8-94ba-958f762b5962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:03.860662 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef683b-e782-40b8-94ba-958f762b5962 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:03.860945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef683b-e782-40b8-94ba-958f762b5962 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:03.861219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef683b-e782-40b8-94ba-958f762b5962 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:03.861941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef683b-e782-40b8-94ba-958f762b5962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:03.862245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef683b-e782-40b8-94ba-958f762b5962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:03.863705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef683b-e782-40b8-94ba-958f762b5962 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:03.878492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 375/763] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:03.884069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda1196-79fe-4010-bf5e-5b91f9cf1af5 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:03.972656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a824a2-35d2-4fcd-9863-70b31fe79e07 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:03.972968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a824a2-35d2-4fcd-9863-70b31fe79e07 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:03.973207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a824a2-35d2-4fcd-9863-70b31fe79e07 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:03.975751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 376/764] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:39:03.979672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deffa06-8f80-41d2-90a8-84e7dce651c4 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:03.980018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deffa06-8f80-41d2-90a8-84e7dce651c4 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:03.980243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deffa06-8f80-41d2-90a8-84e7dce651c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:04.002081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda1196-79fe-4010-bf5e-5b91f9cf1af5 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:04.019987 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-0deffa06-8f80-41d2-90a8-84e7dce651c4 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:04.021186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0deffa06-8f80-41d2-90a8-84e7dce651c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:04.038810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cf7ca-2a9e-4b3e-8e6d-6d0276ab2e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:04.039448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cf7ca-2a9e-4b3e-8e6d-6d0276ab2e5e None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:04.039777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cf7ca-2a9e-4b3e-8e6d-6d0276ab2e5e None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:04.040482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cf7ca-2a9e-4b3e-8e6d-6d0276ab2e5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:04.048275 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cf7ca-2a9e-4b3e-8e6d-6d0276ab2e5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:04.049740 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cf7ca-2a9e-4b3e-8e6d-6d0276ab2e5e None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:04.052459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:04.053791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-425cf7ca-2a9e-4b3e-8e6d-6d0276ab2e5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:04.065330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:04.066257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:04.067225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:04.067225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:04.067225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:04.067225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:04.067225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:04.067225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:04.067225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:04.067225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:04.067225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 389/765] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:04.072310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cf7ca-2a9e-4b3e-8e6d-6d0276ab2e5e None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:04.177643 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cf7ca-2a9e-4b3e-8e6d-6d0276ab2e5e None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:04.186008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 377/766] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:04.213246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3453a5a-209a-4f0d-9f64-2957cf6a448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:04.213943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3453a5a-209a-4f0d-9f64-2957cf6a448f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:04.214231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3453a5a-209a-4f0d-9f64-2957cf6a448f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:04.214509 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3453a5a-209a-4f0d-9f64-2957cf6a448f None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:04.215207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3453a5a-209a-4f0d-9f64-2957cf6a448f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:04.215577 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3453a5a-209a-4f0d-9f64-2957cf6a448f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:04.217146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3453a5a-209a-4f0d-9f64-2957cf6a448f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:04.229603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 390/767] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:04.235953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deffa06-8f80-41d2-90a8-84e7dce651c4 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:04.348324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deffa06-8f80-41d2-90a8-84e7dce651c4 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:04.388017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9b5f3-5075-465a-98be-b13d842cf0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:04.389086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9b5f3-5075-465a-98be-b13d842cf0dc None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:04.389567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9b5f3-5075-465a-98be-b13d842cf0dc None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:04.390063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9b5f3-5075-465a-98be-b13d842cf0dc None admin] PATH_INFO: `/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/3f3efa9f3eb24864943b68d241435817/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:04.395053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9b5f3-5075-465a-98be-b13d842cf0dc None admin] RBAC: Authorizing `identity:create_grant(project_id=12f9875f45a84e92a848b8fc1636d6c6, user_id=3f3efa9f3eb24864943b68d241435817, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:04.396754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9b5f3-5075-465a-98be-b13d842cf0dc None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:04.419267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 378/768] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:39:04 2022] PUT /identity/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/3f3efa9f3eb24864943b68d241435817/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:39:04.607260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3453a5a-209a-4f0d-9f64-2957cf6a448f None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:04.670323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:04.716869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3453a5a-209a-4f0d-9f64-2957cf6a448f None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:04.753917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f50b60-9d50-448d-be28-bd6840c46f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:04.754545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f50b60-9d50-448d-be28-bd6840c46f29 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:04.754814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f50b60-9d50-448d-be28-bd6840c46f29 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:04.755074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f50b60-9d50-448d-be28-bd6840c46f29 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:04.755643 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f50b60-9d50-448d-be28-bd6840c46f29 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:04.757034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f50b60-9d50-448d-be28-bd6840c46f29 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:04.764700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fbf05-2203-4548-bccf-e396fb0aaa09 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:04.765240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fbf05-2203-4548-bccf-e396fb0aaa09 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:04.765332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fbf05-2203-4548-bccf-e396fb0aaa09 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-37f50b60-9d50-448d-be28-bd6840c46f29 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:04.765741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:04.766713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:04.767599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 391/770] 10.222.0.36 () {56 vars in 1129 bytes} [Thu Aug 18 14:39:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:04.768926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 379/770] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:39:04.772060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663fea6-9770-4ab8-a56c-df43b95afdc7 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:04.772334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663fea6-9770-4ab8-a56c-df43b95afdc7 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:04.772443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fbf05-2203-4548-bccf-e396fb0aaa09 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:04.772602 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663fea6-9770-4ab8-a56c-df43b95afdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:04.785430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:04.802521 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-0663fea6-9770-4ab8-a56c-df43b95afdc7 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:04.803731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-0663fea6-9770-4ab8-a56c-df43b95afdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:04.931827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 392/771] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:04.954244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fbf05-2203-4548-bccf-e396fb0aaa09 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:05.009857 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d9af2-f271-408a-a7a7-44d875df7cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:05.010464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d9af2-f271-408a-a7a7-44d875df7cb6 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:05.010739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d9af2-f271-408a-a7a7-44d875df7cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:05.011009 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d9af2-f271-408a-a7a7-44d875df7cb6 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:05.011635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d9af2-f271-408a-a7a7-44d875df7cb6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:05.011958 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d9af2-f271-408a-a7a7-44d875df7cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:05.013366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d9af2-f271-408a-a7a7-44d875df7cb6 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:05.023665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 380/772] 10.222.0.36 () {56 vars in 1124 bytes} [Thu Aug 18 14:39:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:05.028817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663fea6-9770-4ab8-a56c-df43b95afdc7 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:05.088640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:05.145356 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663fea6-9770-4ab8-a56c-df43b95afdc7 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:05.184897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff8842-4563-4407-8fc4-743265595014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:05.185504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff8842-4563-4407-8fc4-743265595014 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:05.185780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff8842-4563-4407-8fc4-743265595014 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:05.186046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff8842-4563-4407-8fc4-743265595014 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:05.186667 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff8842-4563-4407-8fc4-743265595014 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:05.187038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff8842-4563-4407-8fc4-743265595014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:05.188933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff8842-4563-4407-8fc4-743265595014 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:05.194259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 393/773] 10.222.0.36 () {56 vars in 1124 bytes} [Thu Aug 18 14:39:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:05.199714 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d9af2-f271-408a-a7a7-44d875df7cb6 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:05.307492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:05.345248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d9af2-f271-408a-a7a7-44d875df7cb6 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:05.362659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff8842-4563-4407-8fc4-743265595014 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:05.402358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdb0d6-a54b-4b03-9799-0b8d7048a0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:05.403346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdb0d6-a54b-4b03-9799-0b8d7048a0ab None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:05.403900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdb0d6-a54b-4b03-9799-0b8d7048a0ab None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:05.404378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdb0d6-a54b-4b03-9799-0b8d7048a0ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:05.405302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fdb0d6-a54b-4b03-9799-0b8d7048a0ab None admin] RBAC: Adding query filter params (interface=public, service_id=346cb97579e24066932c0c3b590f5bef, region_id=RegionOne) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:05.405866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fdb0d6-a54b-4b03-9799-0b8d7048a0ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:05.407439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fdb0d6-a54b-4b03-9799-0b8d7048a0ab None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:05.419073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 381/774] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Aug 18 14:39:05 2022] GET /identity/v3/endpoints?service_id=346cb97579e24066932c0c3b590f5bef&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:05.424396 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdb0d6-a54b-4b03-9799-0b8d7048a0ab None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:05.543613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff8842-4563-4407-8fc4-743265595014 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:05.562750 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:05.594134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdb0d6-a54b-4b03-9799-0b8d7048a0ab None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:05.598933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12728ad-bce2-4b62-950c-583ec65b52a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:05.599836 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12728ad-bce2-4b62-950c-583ec65b52a0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:05.600318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12728ad-bce2-4b62-950c-583ec65b52a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:05.600807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12728ad-bce2-4b62-950c-583ec65b52a0 None admin] PATH_INFO: `/v3/services/network` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:05.601591 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12728ad-bce2-4b62-950c-583ec65b52a0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:05.603162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12728ad-bce2-4b62-950c-583ec65b52a0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-b12728ad-bce2-4b62-950c-583ec65b52a0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:05.619578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:05.620637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 14:39:05.621487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:05.632640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 394/775] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:05 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:05.635793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12728ad-bce2-4b62-950c-583ec65b52a0 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:05.672303 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b461ad-f420-41d0-9e14-eb5e5dca3ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:05.673277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b461ad-f420-41d0-9e14-eb5e5dca3ea5 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:05.673758 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b461ad-f420-41d0-9e14-eb5e5dca3ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:05.674229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b461ad-f420-41d0-9e14-eb5e5dca3ea5 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:05.675088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b461ad-f420-41d0-9e14-eb5e5dca3ea5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:05.676742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b461ad-f420-41d0-9e14-eb5e5dca3ea5 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:05.684470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:05.694389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 382/776] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Aug 18 14:39:05 2022] GET /identity/v3/services => generated 772 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:05.793118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2a6aa-029b-48c2-a0a9-59c5d2c267c1 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:05.793794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2a6aa-029b-48c2-a0a9-59c5d2c267c1 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:05.793794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2a6aa-029b-48c2-a0a9-59c5d2c267c1 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:05.796448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 383/777] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:39:05.801035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3ff38-fcf2-4921-90f5-d6a575c5ae3b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:05.801323 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3ff38-fcf2-4921-90f5-d6a575c5ae3b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:05.801580 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3ff38-fcf2-4921-90f5-d6a575c5ae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:05.804810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12728ad-bce2-4b62-950c-583ec65b52a0 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:05.835585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-72d3ff38-fcf2-4921-90f5-d6a575c5ae3b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:05.837169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-72d3ff38-fcf2-4921-90f5-d6a575c5ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:05.865243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c83c61-c898-4181-917a-97029f3a527c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:05.865859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c83c61-c898-4181-917a-97029f3a527c None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:05.866133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c83c61-c898-4181-917a-97029f3a527c None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:05.866401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c83c61-c898-4181-917a-97029f3a527c None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:05.867185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c83c61-c898-4181-917a-97029f3a527c None admin] RBAC: Adding query filter params (name=network) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:05.867488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c83c61-c898-4181-917a-97029f3a527c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:05.869076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c83c61-c898-4181-917a-97029f3a527c None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:05.878247 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 395/778] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:39:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:05.883407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c83c61-c898-4181-917a-97029f3a527c None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:05.933399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 384/779] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:05.952095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:06.012301 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c83c61-c898-4181-917a-97029f3a527c None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:06.052472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4b1a9-7095-477e-8b73-82088b0e1b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:06.053012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4b1a9-7095-477e-8b73-82088b0e1b55 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:06.053300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4b1a9-7095-477e-8b73-82088b0e1b55 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:06.053576 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4b1a9-7095-477e-8b73-82088b0e1b55 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:06.054216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4b1a9-7095-477e-8b73-82088b0e1b55 None admin] RBAC: Adding query filter params (type=network) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:06.054520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4b1a9-7095-477e-8b73-82088b0e1b55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:06.055918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4b1a9-7095-477e-8b73-82088b0e1b55 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:06.061256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 396/780] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:39:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:06.066699 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3ff38-fcf2-4921-90f5-d6a575c5ae3b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:06.081838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39b660-c611-405d-b04e-af475e98bab7 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:06.082124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39b660-c611-405d-b04e-af475e98bab7 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:06.082396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39b660-c611-405d-b04e-af475e98bab7 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:06.083910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 397/781] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:06.088796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a169-9724-48fa-a010-35cf643060b1 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:06.089082 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a169-9724-48fa-a010-35cf643060b1 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:06.089416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a169-9724-48fa-a010-35cf643060b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:06.107948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-77b9a169-9724-48fa-a010-35cf643060b1 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:06.109188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-77b9a169-9724-48fa-a010-35cf643060b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:06.221216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 398/782] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:06.255557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3ff38-fcf2-4921-90f5-d6a575c5ae3b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:06.278174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9a169-9724-48fa-a010-35cf643060b1 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:06.298512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:06.319113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d631274-e8f8-4da4-bc64-ac1da5b2e873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:06.319655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d631274-e8f8-4da4-bc64-ac1da5b2e873 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:06.319955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d631274-e8f8-4da4-bc64-ac1da5b2e873 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:06.320196 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:06.320300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d631274-e8f8-4da4-bc64-ac1da5b2e873 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:06.321097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d631274-e8f8-4da4-bc64-ac1da5b2e873 None admin] RBAC: Adding query filter params (interface=public, service_id=d152d36597d04410ad8c8a1a16d6d37e, region_id=RegionOne) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:06.321212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d631274-e8f8-4da4-bc64-ac1da5b2e873 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:06.322741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d631274-e8f8-4da4-bc64-ac1da5b2e873 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:06.329383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 385/783] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Aug 18 14:39:06 2022] GET /identity/v3/endpoints?service_id=d152d36597d04410ad8c8a1a16d6d37e&interface=public®ion_id=RegionOne => generated 194 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:06.334246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d631274-e8f8-4da4-bc64-ac1da5b2e873 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:06.441377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9a169-9724-48fa-a010-35cf643060b1 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:06.504820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d631274-e8f8-4da4-bc64-ac1da5b2e873 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:06.506157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bc2b4-789d-4ffc-b409-a7f40289ece7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:06.506791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bc2b4-789d-4ffc-b409-a7f40289ece7 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:06.507061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bc2b4-789d-4ffc-b409-a7f40289ece7 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:06.507321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bc2b4-789d-4ffc-b409-a7f40289ece7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:06.520955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bc2b4-789d-4ffc-b409-a7f40289ece7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-090bc2b4-789d-4ffc-b409-a7f40289ece7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:06.526413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:06.527358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:06.528254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:06.528254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:06.528254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:06.528254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:06.528254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:06.528254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:06.528254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:06.528254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:06.528254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:06.528254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:06.528254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:06.528254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:06.531029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 399/784] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:39:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:06.532137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bc2b4-789d-4ffc-b409-a7f40289ece7 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:06.548132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431ad06-2e71-4ef4-a9a0-2b95c220771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:06.548741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431ad06-2e71-4ef4-a9a0-2b95c220771f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:06.549015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431ad06-2e71-4ef4-a9a0-2b95c220771f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:06.549282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431ad06-2e71-4ef4-a9a0-2b95c220771f None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:06.549887 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431ad06-2e71-4ef4-a9a0-2b95c220771f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:06.551243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431ad06-2e71-4ef4-a9a0-2b95c220771f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:06.557308 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 386/785] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Aug 18 14:39:06 2022] GET /identity/v3/services => generated 772 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:06.558172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431ad06-2e71-4ef4-a9a0-2b95c220771f None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:06.658819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bc2b4-789d-4ffc-b409-a7f40289ece7 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:06.660689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:06.699010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b84e30-6567-4031-8620-c06efb96df65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:06.699890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b84e30-6567-4031-8620-c06efb96df65 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:06.700172 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b84e30-6567-4031-8620-c06efb96df65 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:06.700447 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b84e30-6567-4031-8620-c06efb96df65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:06.713558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b84e30-6567-4031-8620-c06efb96df65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-68b84e30-6567-4031-8620-c06efb96df65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:06.719355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:06.720354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:06.721221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:06.721221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:06.721221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:06.721221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:06.721221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:06.721221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:06.721221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:06.721221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:06.721221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:06.721221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:06.721221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:39:06.721221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:06.721859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 400/786] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:39:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:06.729160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b84e30-6567-4031-8620-c06efb96df65 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:06.733829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:06.797910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431ad06-2e71-4ef4-a9a0-2b95c220771f None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:06.839880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd13d9-c4be-4bf0-84c3-7a7e1e8e9e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:06.840583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd13d9-c4be-4bf0-84c3-7a7e1e8e9e68 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:06.840864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd13d9-c4be-4bf0-84c3-7a7e1e8e9e68 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:06.841183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd13d9-c4be-4bf0-84c3-7a7e1e8e9e68 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:06.841806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd13d9-c4be-4bf0-84c3-7a7e1e8e9e68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:06.842105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd13d9-c4be-4bf0-84c3-7a7e1e8e9e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:06.842209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b84e30-6567-4031-8620-c06efb96df65 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:06.844166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd13d9-c4be-4bf0-84c3-7a7e1e8e9e68 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:06.863807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 387/787] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:39:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:06.865193 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd13d9-c4be-4bf0-84c3-7a7e1e8e9e68 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:06.891393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d53e4-16d9-41b1-b449-41b02aad2d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:06.892054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d53e4-16d9-41b1-b449-41b02aad2d31 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:06.895144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d53e4-16d9-41b1-b449-41b02aad2d31 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:06.895144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d53e4-16d9-41b1-b449-41b02aad2d31 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:06.895144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d53e4-16d9-41b1-b449-41b02aad2d31 None admin] RBAC: Adding query filter params (name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:06.895144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d53e4-16d9-41b1-b449-41b02aad2d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:06.895144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d53e4-16d9-41b1-b449-41b02aad2d31 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:06.904224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 401/788] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:39:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:06.909691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d53e4-16d9-41b1-b449-41b02aad2d31 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:06.931587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:07.038914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d53e4-16d9-41b1-b449-41b02aad2d31 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:07.040651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd13d9-c4be-4bf0-84c3-7a7e1e8e9e68 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:07.062731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:07.078965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bb9e3-ee6f-4131-9b46-bfcf71cf9732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:07.079641 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bb9e3-ee6f-4131-9b46-bfcf71cf9732 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:07.079980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bb9e3-ee6f-4131-9b46-bfcf71cf9732 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:07.080257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bb9e3-ee6f-4131-9b46-bfcf71cf9732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:07.085142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce49a5-2622-4ea4-a824-824eb57a17f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:07.086015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce49a5-2622-4ea4-a824-824eb57a17f1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:07.086493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce49a5-2622-4ea4-a824-824eb57a17f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:07.086963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce49a5-2622-4ea4-a824-824eb57a17f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:07.091338 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bb9e3-ee6f-4131-9b46-bfcf71cf9732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:07.092910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bb9e3-ee6f-4131-9b46-bfcf71cf9732 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:07.099147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce49a5-2622-4ea4-a824-824eb57a17f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:07.100959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce49a5-2622-4ea4-a824-824eb57a17f1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-6b5bb9e3-ee6f-4131-9b46-bfcf71cf9732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:07.107697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:07.108648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:07.109657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:07.109657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:07.109657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:07.109657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 388/789] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:07.110354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b877a-7ba3-45f9-bdbd-bc6f290e012b None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:07.110609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b877a-7ba3-45f9-bdbd-bc6f290e012b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:07.110886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b877a-7ba3-45f9-bdbd-bc6f290e012b None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:07.116537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 389/790] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:07.116537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b877a-7ba3-45f9-bdbd-bc6f290e012b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-7cce49a5-2622-4ea4-a824-824eb57a17f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:07.125901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:07.126833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:07.127744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:07.127744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:07.127744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:07.134214 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 402/791] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:07.135844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eea0e0-dfdb-48f6-9e92-f7b75ade9bac None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:07.136408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eea0e0-dfdb-48f6-9e92-f7b75ade9bac None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:07.136902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eea0e0-dfdb-48f6-9e92-f7b75ade9bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:07.157163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-c4eea0e0-dfdb-48f6-9e92-f7b75ade9bac None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:07.158651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c4eea0e0-dfdb-48f6-9e92-f7b75ade9bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:07.225071 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:07.236037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 403/792] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:07.237363 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eea0e0-dfdb-48f6-9e92-f7b75ade9bac None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:07.245020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b877a-7ba3-45f9-bdbd-bc6f290e012b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:07.325106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547001-630f-4626-a9ff-a517d8081244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:07.326127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547001-630f-4626-a9ff-a517d8081244 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:07.326610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547001-630f-4626-a9ff-a517d8081244 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:07.327082 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547001-630f-4626-a9ff-a517d8081244 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:07.328464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547001-630f-4626-a9ff-a517d8081244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:07.328969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547001-630f-4626-a9ff-a517d8081244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:07.330558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547001-630f-4626-a9ff-a517d8081244 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:07.353734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eea0e0-dfdb-48f6-9e92-f7b75ade9bac None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:07.362129 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 390/793] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:07.378499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547001-630f-4626-a9ff-a517d8081244 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:07.398760 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c47500-060f-448a-b734-c98e0cd97988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:07.399761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c47500-060f-448a-b734-c98e0cd97988 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:07.400300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c47500-060f-448a-b734-c98e0cd97988 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:07.405045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c47500-060f-448a-b734-c98e0cd97988 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:07.406480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c47500-060f-448a-b734-c98e0cd97988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:07.407201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c47500-060f-448a-b734-c98e0cd97988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:07.409188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c47500-060f-448a-b734-c98e0cd97988 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:07.431392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:07.435142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 404/794] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:07.443042 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c47500-060f-448a-b734-c98e0cd97988 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:07.548056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:07.549421 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547001-630f-4626-a9ff-a517d8081244 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:07.617066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c47500-060f-448a-b734-c98e0cd97988 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:07.657323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509790f5-56af-48f0-b787-e9076e0a667b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:07.658431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509790f5-56af-48f0-b787-e9076e0a667b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:07.658915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509790f5-56af-48f0-b787-e9076e0a667b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:07.659389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509790f5-56af-48f0-b787-e9076e0a667b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:07.668656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509790f5-56af-48f0-b787-e9076e0a667b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:07.670449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509790f5-56af-48f0-b787-e9076e0a667b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-509790f5-56af-48f0-b787-e9076e0a667b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:07.682948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:07.683987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:07.684936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:07.684936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:07.684936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:07.684936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:07.684936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:07.684936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:07.684936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:07.684936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:07.684936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 14:39:07.684936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:07.688134 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 405/795] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Aug 18 14:39:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:07.689196 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509790f5-56af-48f0-b787-e9076e0a667b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:07.725431 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0633ff-7b18-475c-b788-9a4c0f972290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:07.726063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0633ff-7b18-475c-b788-9a4c0f972290 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:07.726338 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0633ff-7b18-475c-b788-9a4c0f972290 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:07.726609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0633ff-7b18-475c-b788-9a4c0f972290 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:07.741847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0633ff-7b18-475c-b788-9a4c0f972290 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:07.744661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0633ff-7b18-475c-b788-9a4c0f972290 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-dd0633ff-7b18-475c-b788-9a4c0f972290 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:07.762186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:07.763366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:07.764568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:07.764568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:07.764568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:07.764568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:07.764568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:07.764568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:07.764568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:07.764568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:07.764568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 14:39:07.764568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:07.767611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 391/796] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:39:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:07.769179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0633ff-7b18-475c-b788-9a4c0f972290 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:07.802916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509790f5-56af-48f0-b787-e9076e0a667b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:07.824681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:07.842220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ec25e-13d1-4ae4-8364-ab2e4ef7d899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:07.843061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ec25e-13d1-4ae4-8364-ab2e4ef7d899 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:07.843469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ec25e-13d1-4ae4-8364-ab2e4ef7d899 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:07.843867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ec25e-13d1-4ae4-8364-ab2e4ef7d899 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:07.855445 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ec25e-13d1-4ae4-8364-ab2e4ef7d899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-c37ec25e-13d1-4ae4-8364-ab2e4ef7d899 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:07.861400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:07.862541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:07.863541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:07.863541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:07.863541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:07.863541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:07.863541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:07.863541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:07.863541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:07.863541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:07.863541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:07.863541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:07.863541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:07.863541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:07.865493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 406/797] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:39:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:07.866403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ec25e-13d1-4ae4-8364-ab2e4ef7d899 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:07.908849 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:07.926433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0633ff-7b18-475c-b788-9a4c0f972290 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:07.963209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128cb9d-fe6f-461c-8c73-a3d60e5b08c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:07.963832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128cb9d-fe6f-461c-8c73-a3d60e5b08c0 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:07.964103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128cb9d-fe6f-461c-8c73-a3d60e5b08c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:07.964367 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128cb9d-fe6f-461c-8c73-a3d60e5b08c0 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:07.965074 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128cb9d-fe6f-461c-8c73-a3d60e5b08c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:07.965381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128cb9d-fe6f-461c-8c73-a3d60e5b08c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:07.966808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128cb9d-fe6f-461c-8c73-a3d60e5b08c0 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:07.984621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 392/798] 10.222.0.36 () {56 vars in 1149 bytes} [Thu Aug 18 14:39:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:07.985440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128cb9d-fe6f-461c-8c73-a3d60e5b08c0 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:07.991013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ec25e-13d1-4ae4-8364-ab2e4ef7d899 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:08.030112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46201731-0f96-47a8-92a2-55ee2abad23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:08.030751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46201731-0f96-47a8-92a2-55ee2abad23a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.031401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46201731-0f96-47a8-92a2-55ee2abad23a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.031714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46201731-0f96-47a8-92a2-55ee2abad23a None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.032708 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46201731-0f96-47a8-92a2-55ee2abad23a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:08.033013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46201731-0f96-47a8-92a2-55ee2abad23a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:08.035152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46201731-0f96-47a8-92a2-55ee2abad23a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:08.063420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 407/799] 10.222.0.36 () {56 vars in 1135 bytes} [Thu Aug 18 14:39:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:08.063420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717de77e-0698-4584-b5f4-ea691f0288af None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.063420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717de77e-0698-4584-b5f4-ea691f0288af None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.063420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717de77e-0698-4584-b5f4-ea691f0288af None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.063420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 408/800] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:08.063420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717de77e-0698-4584-b5f4-ea691f0288af None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:08.091665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128cb9d-fe6f-461c-8c73-a3d60e5b08c0 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:08.105784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:08.134922 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbc55e-a0df-492f-bf11-186d2115ea02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:08.135671 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbc55e-a0df-492f-bf11-186d2115ea02 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.136744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbc55e-a0df-492f-bf11-186d2115ea02 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.137026 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbc55e-a0df-492f-bf11-186d2115ea02 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.137691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbc55e-a0df-492f-bf11-186d2115ea02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:08.137993 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbc55e-a0df-492f-bf11-186d2115ea02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:08.139347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbc55e-a0df-492f-bf11-186d2115ea02 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:08.154245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 393/801] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:39:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:08.154245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbc55e-a0df-492f-bf11-186d2115ea02 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:08.165986 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717de77e-0698-4584-b5f4-ea691f0288af None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:08.172337 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:08.201265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469fb00-e8bd-451d-bec3-b98599cf9e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:08.201887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469fb00-e8bd-451d-bec3-b98599cf9e95 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.202157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469fb00-e8bd-451d-bec3-b98599cf9e95 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.203133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469fb00-e8bd-451d-bec3-b98599cf9e95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.213529 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469fb00-e8bd-451d-bec3-b98599cf9e95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:08.215111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469fb00-e8bd-451d-bec3-b98599cf9e95 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-9469fb00-e8bd-451d-bec3-b98599cf9e95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:08.229882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:08.230859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:08.232200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:08.232200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:08.232200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:08.232200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 409/802] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:08.232200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb03ce-090d-4ea2-9a19-d9024aedc67d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.232498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb03ce-090d-4ea2-9a19-d9024aedc67d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.232757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb03ce-090d-4ea2-9a19-d9024aedc67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.250151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-8dbb03ce-090d-4ea2-9a19-d9024aedc67d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:08.251328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8dbb03ce-090d-4ea2-9a19-d9024aedc67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:08.266138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbc55e-a0df-492f-bf11-186d2115ea02 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:08.299542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a87dd-c1a3-42a8-b9df-d08272ff8158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:08.300202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a87dd-c1a3-42a8-b9df-d08272ff8158 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.300482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a87dd-c1a3-42a8-b9df-d08272ff8158 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.300756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a87dd-c1a3-42a8-b9df-d08272ff8158 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.311209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a87dd-c1a3-42a8-b9df-d08272ff8158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:08.312697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a87dd-c1a3-42a8-b9df-d08272ff8158 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-f60a87dd-c1a3-42a8-b9df-d08272ff8158 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:08.327290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:08.328961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:08.329858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:08.329858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:08.329858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:08.330069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 410/803] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:08.330069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb03ce-090d-4ea2-9a19-d9024aedc67d None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:08.332852 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 394/804] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:08.333681 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a87dd-c1a3-42a8-b9df-d08272ff8158 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:08.358871 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:08.406225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:08.458126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb03ce-090d-4ea2-9a19-d9024aedc67d None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:08.458720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a87dd-c1a3-42a8-b9df-d08272ff8158 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:08.496151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15144f31-c80a-4b61-a4e5-55dd0b347f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:08.496813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15144f31-c80a-4b61-a4e5-55dd0b347f8c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.497102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15144f31-c80a-4b61-a4e5-55dd0b347f8c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.497362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15144f31-c80a-4b61-a4e5-55dd0b347f8c None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.498046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15144f31-c80a-4b61-a4e5-55dd0b347f8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:08.498354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15144f31-c80a-4b61-a4e5-55dd0b347f8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:08.500570 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15144f31-c80a-4b61-a4e5-55dd0b347f8c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:08.501457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca76399-35a5-4bab-823c-b5a2adf04f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:08.502570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca76399-35a5-4bab-823c-b5a2adf04f54 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.503641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca76399-35a5-4bab-823c-b5a2adf04f54 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.504158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca76399-35a5-4bab-823c-b5a2adf04f54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.513313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 395/805] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:08.516713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca76399-35a5-4bab-823c-b5a2adf04f54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:08.518416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca76399-35a5-4bab-823c-b5a2adf04f54 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:08.518781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15144f31-c80a-4b61-a4e5-55dd0b347f8c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-9ca76399-35a5-4bab-823c-b5a2adf04f54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:08.537901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:08.538921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:08.539853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:08.539853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:08.539853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:08.544385 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 411/806] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:08.545023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca76399-35a5-4bab-823c-b5a2adf04f54 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:08.621146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:08.635751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15144f31-c80a-4b61-a4e5-55dd0b347f8c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:08.652138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca76399-35a5-4bab-823c-b5a2adf04f54 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:08.660551 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:08.673043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c44cdd-b8ac-49af-9c81-437f1b09d192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:08.673643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c44cdd-b8ac-49af-9c81-437f1b09d192 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.673917 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c44cdd-b8ac-49af-9c81-437f1b09d192 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.674273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c44cdd-b8ac-49af-9c81-437f1b09d192 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.674809 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c44cdd-b8ac-49af-9c81-437f1b09d192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:08.675103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c44cdd-b8ac-49af-9c81-437f1b09d192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:08.676499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c44cdd-b8ac-49af-9c81-437f1b09d192 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:08.687496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 396/807] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:08.687807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492eb1b-b3f5-4e68-8d43-f7299b4dc815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:08.688302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c44cdd-b8ac-49af-9c81-437f1b09d192 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:08.688488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492eb1b-b3f5-4e68-8d43-f7299b4dc815 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.688722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492eb1b-b3f5-4e68-8d43-f7299b4dc815 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.688993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492eb1b-b3f5-4e68-8d43-f7299b4dc815 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.698355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492eb1b-b3f5-4e68-8d43-f7299b4dc815 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:08.699827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492eb1b-b3f5-4e68-8d43-f7299b4dc815 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-2492eb1b-b3f5-4e68-8d43-f7299b4dc815 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:08.711345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:08.712369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:08.713344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:08.713344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:08.713344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:08.713344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:08.713344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:08.713344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:08.713344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:08.713344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:08.713344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 412/808] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:08.713833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492eb1b-b3f5-4e68-8d43-f7299b4dc815 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:08.790365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c44cdd-b8ac-49af-9c81-437f1b09d192 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:08.810272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492eb1b-b3f5-4e68-8d43-f7299b4dc815 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:08.823456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25224b6f-257b-4dff-8cfd-49b5bfb064d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:08.824277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25224b6f-257b-4dff-8cfd-49b5bfb064d6 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.824547 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25224b6f-257b-4dff-8cfd-49b5bfb064d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.824811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25224b6f-257b-4dff-8cfd-49b5bfb064d6 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.825433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25224b6f-257b-4dff-8cfd-49b5bfb064d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:08.825725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25224b6f-257b-4dff-8cfd-49b5bfb064d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:08.827101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25224b6f-257b-4dff-8cfd-49b5bfb064d6 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:08.838366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 397/809] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:08.839816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943849e-1c95-4fea-bff1-d769e867a017 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.840094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943849e-1c95-4fea-bff1-d769e867a017 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.840361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943849e-1c95-4fea-bff1-d769e867a017 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.841526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 398/810] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:08.842954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943849e-1c95-4fea-bff1-d769e867a017 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:08.843379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:08.849133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f9830-8943-4b3a-b989-114bf7310979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:08.849759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f9830-8943-4b3a-b989-114bf7310979 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.850031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f9830-8943-4b3a-b989-114bf7310979 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.850300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f9830-8943-4b3a-b989-114bf7310979 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.851284 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f9830-8943-4b3a-b989-114bf7310979 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:08.852673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f9830-8943-4b3a-b989-114bf7310979 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-3e3f9830-8943-4b3a-b989-114bf7310979 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:08.862090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:08.863220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:08.864126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 413/811] 10.222.0.36 () {56 vars in 1129 bytes} [Thu Aug 18 14:39:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:08.864918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f9830-8943-4b3a-b989-114bf7310979 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:08.888967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:08.945568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943849e-1c95-4fea-bff1-d769e867a017 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:08.964867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f9830-8943-4b3a-b989-114bf7310979 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:08.978714 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73775f68-86c1-485d-9dab-1006f38da1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:08.979324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73775f68-86c1-485d-9dab-1006f38da1dc None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.979600 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73775f68-86c1-485d-9dab-1006f38da1dc None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.979907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73775f68-86c1-485d-9dab-1006f38da1dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.986910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73775f68-86c1-485d-9dab-1006f38da1dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:08.988386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73775f68-86c1-485d-9dab-1006f38da1dc None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:08.997673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e329832-f0ad-42b7-826c-1c18d0e8dbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:08.998272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e329832-f0ad-42b7-826c-1c18d0e8dbfa None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:08.998551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e329832-f0ad-42b7-826c-1c18d0e8dbfa None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:08.999076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e329832-f0ad-42b7-826c-1c18d0e8dbfa None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-73775f68-86c1-485d-9dab-1006f38da1dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:08.999248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:09.000704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:09.001602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:09.001602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:09.001602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:09.001602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:09.001602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:09.001602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:09.001602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:09.001602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:09.002040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e329832-f0ad-42b7-826c-1c18d0e8dbfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:09.002040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e329832-f0ad-42b7-826c-1c18d0e8dbfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:09.002173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 399/812] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:09.002173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73775f68-86c1-485d-9dab-1006f38da1dc None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:09.002297 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e329832-f0ad-42b7-826c-1c18d0e8dbfa None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:09.013402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 414/813] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:09.014532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29872d23-ee95-4c84-ab64-ad3a4d3077c0 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:09.014814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29872d23-ee95-4c84-ab64-ad3a4d3077c0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:09.015122 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29872d23-ee95-4c84-ab64-ad3a4d3077c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:09.032305 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-29872d23-ee95-4c84-ab64-ad3a4d3077c0 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:09.033488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-29872d23-ee95-4c84-ab64-ad3a4d3077c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:09.082390 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:09.099729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 415/814] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:09.100536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29872d23-ee95-4c84-ab64-ad3a4d3077c0 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:09.111073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73775f68-86c1-485d-9dab-1006f38da1dc None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:09.114829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:09.147728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffe5f0-35fa-488e-906c-509848a78d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:09.148580 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffe5f0-35fa-488e-906c-509848a78d31 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:09.148866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffe5f0-35fa-488e-906c-509848a78d31 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:09.149352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffe5f0-35fa-488e-906c-509848a78d31 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:09.158206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffe5f0-35fa-488e-906c-509848a78d31 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:09.159633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffe5f0-35fa-488e-906c-509848a78d31 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-2bffe5f0-35fa-488e-906c-509848a78d31 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:09.170375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:09.172347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:09.173654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:09.173654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:09.173654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:09.173654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:09.173654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:09.173654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:09.173654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:09.173654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:09.173654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:39:09.173654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:09.173654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 400/815] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Aug 18 14:39:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:09.173654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffe5f0-35fa-488e-906c-509848a78d31 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:09.211130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29872d23-ee95-4c84-ab64-ad3a4d3077c0 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:09.246086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddaf5db-4474-44ae-93a1-cf62e40e6bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:09.246691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddaf5db-4474-44ae-93a1-cf62e40e6bab None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:09.246970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddaf5db-4474-44ae-93a1-cf62e40e6bab None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:09.247241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddaf5db-4474-44ae-93a1-cf62e40e6bab None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:09.247896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddaf5db-4474-44ae-93a1-cf62e40e6bab None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:09.248202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddaf5db-4474-44ae-93a1-cf62e40e6bab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:09.250372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddaf5db-4474-44ae-93a1-cf62e40e6bab None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:09.254712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 416/816] 10.222.0.36 () {56 vars in 1124 bytes} [Thu Aug 18 14:39:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:09.255462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddaf5db-4474-44ae-93a1-cf62e40e6bab None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:09.286320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffe5f0-35fa-488e-906c-509848a78d31 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:09.311456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:09.326214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1170ca-31cb-479b-ab8d-094725701810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:09.327089 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1170ca-31cb-479b-ab8d-094725701810 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:09.328219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1170ca-31cb-479b-ab8d-094725701810 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:09.328668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1170ca-31cb-479b-ab8d-094725701810 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:09.329520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1170ca-31cb-479b-ab8d-094725701810 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:09.330019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1170ca-31cb-479b-ab8d-094725701810 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:09.331628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1170ca-31cb-479b-ab8d-094725701810 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:09.344193 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 401/817] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:09.345082 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1170ca-31cb-479b-ab8d-094725701810 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:09.367356 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddaf5db-4474-44ae-93a1-cf62e40e6bab None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:09.371253 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:09.402213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ba112-6fdf-40c8-81b3-e4195d6a7057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:09.402857 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ba112-6fdf-40c8-81b3-e4195d6a7057 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:09.403133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ba112-6fdf-40c8-81b3-e4195d6a7057 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:09.403400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ba112-6fdf-40c8-81b3-e4195d6a7057 None admin] PATH_INFO: `/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/cbb13837e1694d7aa928ac855e21d1a9/roles/fb5e46782bfc4b09b89697a6b033fecc` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:09.416957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ba112-6fdf-40c8-81b3-e4195d6a7057 None admin] RBAC: Authorizing `identity:create_grant(project_id=12f9875f45a84e92a848b8fc1636d6c6, user_id=cbb13837e1694d7aa928ac855e21d1a9, role_id=fb5e46782bfc4b09b89697a6b033fecc)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:09.418379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ba112-6fdf-40c8-81b3-e4195d6a7057 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:09.431179 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 417/818] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:39:09 2022] PUT /identity/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/cbb13837e1694d7aa928ac855e21d1a9/roles/fb5e46782bfc4b09b89697a6b033fecc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:39:09.431997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ba112-6fdf-40c8-81b3-e4195d6a7057 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:09.450405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1170ca-31cb-479b-ab8d-094725701810 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:09.490092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa3634-9ebb-4016-adcd-1d460e112a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:09.490713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa3634-9ebb-4016-adcd-1d460e112a0b None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:09.491007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa3634-9ebb-4016-adcd-1d460e112a0b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:09.491257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa3634-9ebb-4016-adcd-1d460e112a0b None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:09.492075 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa3634-9ebb-4016-adcd-1d460e112a0b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:09.492355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa3634-9ebb-4016-adcd-1d460e112a0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:09.493877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa3634-9ebb-4016-adcd-1d460e112a0b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:09.514568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 402/819] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Aug 18 14:39:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:09.515473 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa3634-9ebb-4016-adcd-1d460e112a0b None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:09.549502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:09.553946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ba112-6fdf-40c8-81b3-e4195d6a7057 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:09.597365 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb423050-4b6b-407c-bf05-423315d7a898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:09.597977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb423050-4b6b-407c-bf05-423315d7a898 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:09.598260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb423050-4b6b-407c-bf05-423315d7a898 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:09.598533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb423050-4b6b-407c-bf05-423315d7a898 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:09.599166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb423050-4b6b-407c-bf05-423315d7a898 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:09.599470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb423050-4b6b-407c-bf05-423315d7a898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:09.600992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb423050-4b6b-407c-bf05-423315d7a898 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:09.611631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 418/820] 10.222.0.36 () {56 vars in 1124 bytes} [Thu Aug 18 14:39:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:09.611842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:09.611941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb423050-4b6b-407c-bf05-423315d7a898 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:09.634875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa3634-9ebb-4016-adcd-1d460e112a0b None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:09.677271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab434a-07be-4f30-8722-4b03a86bea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:09.677927 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab434a-07be-4f30-8722-4b03a86bea36 None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:09.678210 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab434a-07be-4f30-8722-4b03a86bea36 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:09.678479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab434a-07be-4f30-8722-4b03a86bea36 None admin] PATH_INFO: `/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/f578727b96564c3db6b5a1e6cac109d8/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:09.680426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab434a-07be-4f30-8722-4b03a86bea36 None admin] RBAC: Authorizing `identity:create_grant(project_id=12f9875f45a84e92a848b8fc1636d6c6, user_id=f578727b96564c3db6b5a1e6cac109d8, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:09.681856 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab434a-07be-4f30-8722-4b03a86bea36 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:09.703819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 403/821] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:39:09 2022] PUT /identity/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/f578727b96564c3db6b5a1e6cac109d8/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:39:09.706030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab434a-07be-4f30-8722-4b03a86bea36 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:09.720781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb423050-4b6b-407c-bf05-423315d7a898 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:09.759855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfba738-2b91-4e83-9219-1303ad926b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:09.760618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfba738-2b91-4e83-9219-1303ad926b93 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:09.761007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfba738-2b91-4e83-9219-1303ad926b93 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:09.761393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfba738-2b91-4e83-9219-1303ad926b93 None admin] PATH_INFO: `/v3/services/network` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:09.761985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfba738-2b91-4e83-9219-1303ad926b93 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:09.763544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfba738-2b91-4e83-9219-1303ad926b93 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-0cfba738-2b91-4e83-9219-1303ad926b93 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:09.777751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:09.778813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 14:39:09.779659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:09.788356 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 419/822] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:09 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:09.789746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfba738-2b91-4e83-9219-1303ad926b93 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:09.823225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:09.842936 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab434a-07be-4f30-8722-4b03a86bea36 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:09.884183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502128f-95f7-463d-8653-b8df4066ab9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:09.884797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502128f-95f7-463d-8653-b8df4066ab9e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:09.885067 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502128f-95f7-463d-8653-b8df4066ab9e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:09.885501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502128f-95f7-463d-8653-b8df4066ab9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:09.908674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:09.909646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfba738-2b91-4e83-9219-1303ad926b93 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:09.954933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b5d9e-ac24-4325-93cd-fb85ed2f0c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:09.955478 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b5d9e-ac24-4325-93cd-fb85ed2f0c29 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:09.955936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b5d9e-ac24-4325-93cd-fb85ed2f0c29 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:09.956337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b5d9e-ac24-4325-93cd-fb85ed2f0c29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:09.957011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b5d9e-ac24-4325-93cd-fb85ed2f0c29 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:09.959289 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b5d9e-ac24-4325-93cd-fb85ed2f0c29 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:09.974590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502128f-95f7-463d-8653-b8df4066ab9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:09.976356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502128f-95f7-463d-8653-b8df4066ab9e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:09.988507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 420/823] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:39:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:09.989558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b5d9e-ac24-4325-93cd-fb85ed2f0c29 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:10.089183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 404/824] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:39:10.094669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502128f-95f7-463d-8653-b8df4066ab9e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:10.095369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:10.144599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:10.155396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b5d9e-ac24-4325-93cd-fb85ed2f0c29 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:10.209631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502128f-95f7-463d-8653-b8df4066ab9e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:10.231281 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfe1a3-a58c-43cb-844c-e541127652b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:10.232070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfe1a3-a58c-43cb-844c-e541127652b0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:10.232353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfe1a3-a58c-43cb-844c-e541127652b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:10.232639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfe1a3-a58c-43cb-844c-e541127652b0 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:10.233376 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfe1a3-a58c-43cb-844c-e541127652b0 None admin] RBAC: Adding query filter params (name=network) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:10.234050 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfe1a3-a58c-43cb-844c-e541127652b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:10.237126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfe1a3-a58c-43cb-844c-e541127652b0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:10.242664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 421/825] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:39:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:10.248241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfe1a3-a58c-43cb-844c-e541127652b0 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:10.249746 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eaf71-3007-48b0-b73f-9d234aee8b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:10.250377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eaf71-3007-48b0-b73f-9d234aee8b3b None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:10.250649 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eaf71-3007-48b0-b73f-9d234aee8b3b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:10.250914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eaf71-3007-48b0-b73f-9d234aee8b3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:10.265587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8eaf71-3007-48b0-b73f-9d234aee8b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:10.267787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8eaf71-3007-48b0-b73f-9d234aee8b3b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-be8eaf71-3007-48b0-b73f-9d234aee8b3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:10.286871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:10.289045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:10.291173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:10.291173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:10.291173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:10.291173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 405/826] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:10.295320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eaf71-3007-48b0-b73f-9d234aee8b3b None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:10.366376 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfe1a3-a58c-43cb-844c-e541127652b0 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:10.400316 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:10.406643 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21a5bf-168a-49dc-9a04-aed991442a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:10.407534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21a5bf-168a-49dc-9a04-aed991442a3b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:10.407968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eaf71-3007-48b0-b73f-9d234aee8b3b None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:10.408569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21a5bf-168a-49dc-9a04-aed991442a3b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:10.408998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21a5bf-168a-49dc-9a04-aed991442a3b None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:10.410101 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21a5bf-168a-49dc-9a04-aed991442a3b None admin] RBAC: Adding query filter params (type=network) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:10.410573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21a5bf-168a-49dc-9a04-aed991442a3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:10.412367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21a5bf-168a-49dc-9a04-aed991442a3b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:10.418026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 422/827] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:39:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:10.423951 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21a5bf-168a-49dc-9a04-aed991442a3b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:10.436325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:10.452094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1f4ff-9b16-4eb4-87a8-a0f80b544b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:10.454077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f4ff-9b16-4eb4-87a8-a0f80b544b97 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:10.454669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f4ff-9b16-4eb4-87a8-a0f80b544b97 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:10.455533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f4ff-9b16-4eb4-87a8-a0f80b544b97 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:10.456587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1f4ff-9b16-4eb4-87a8-a0f80b544b97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:10.456890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1f4ff-9b16-4eb4-87a8-a0f80b544b97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:10.458921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1f4ff-9b16-4eb4-87a8-a0f80b544b97 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:10.477152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 406/828] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:10.484735 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1f4ff-9b16-4eb4-87a8-a0f80b544b97 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:10.582935 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21a5bf-168a-49dc-9a04-aed991442a3b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:10.645955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e68fc-6851-48b9-a064-3b711129f46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:10.646946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e68fc-6851-48b9-a064-3b711129f46b None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:10.647642 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e68fc-6851-48b9-a064-3b711129f46b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:10.648887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e68fc-6851-48b9-a064-3b711129f46b None admin] PATH_INFO: `/v3/endpoints` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:10.650896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e68fc-6851-48b9-a064-3b711129f46b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:10.654199 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e68fc-6851-48b9-a064-3b711129f46b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:10.682161 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1f4ff-9b16-4eb4-87a8-a0f80b544b97 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:10.693890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 423/829] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:39:10 2022] POST /identity/v3/endpoints => generated 337 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:39:10.736879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dac4c-a4b0-49c2-bc73-714f93a68587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:10.737868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dac4c-a4b0-49c2-bc73-714f93a68587 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:10.738357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dac4c-a4b0-49c2-bc73-714f93a68587 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:10.738864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dac4c-a4b0-49c2-bc73-714f93a68587 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:10.747004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dac4c-a4b0-49c2-bc73-714f93a68587 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:10.749176 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dac4c-a4b0-49c2-bc73-714f93a68587 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-076dac4c-a4b0-49c2-bc73-714f93a68587 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:10.762063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:10.763077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:10.764044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:10.764044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:10.764044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:10.764044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:10.764044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:10.764044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:10.764044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:10.764044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:10.766593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 407/830] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:10.772082 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e68fc-6851-48b9-a064-3b711129f46b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:10.845814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:10.892617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e68fc-6851-48b9-a064-3b711129f46b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:10.936760 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efb5d9-9e9e-41df-9740-448a27e27c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:10.938031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efb5d9-9e9e-41df-9740-448a27e27c90 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:10.938313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efb5d9-9e9e-41df-9740-448a27e27c90 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:10.938586 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efb5d9-9e9e-41df-9740-448a27e27c90 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:10.939287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efb5d9-9e9e-41df-9740-448a27e27c90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:10.939593 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efb5d9-9e9e-41df-9740-448a27e27c90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:10.941099 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efb5d9-9e9e-41df-9740-448a27e27c90 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:10.955024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 424/831] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:10.960643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dac4c-a4b0-49c2-bc73-714f93a68587 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:10.975052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:11.083350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dac4c-a4b0-49c2-bc73-714f93a68587 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:11.125971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7614404-b4ea-4f45-9d59-3c61c495fa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:11.126579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7614404-b4ea-4f45-9d59-3c61c495fa46 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:11.126858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7614404-b4ea-4f45-9d59-3c61c495fa46 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:11.127132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7614404-b4ea-4f45-9d59-3c61c495fa46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:11.127886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7614404-b4ea-4f45-9d59-3c61c495fa46 None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=3f3efa9f3eb24864943b68d241435817) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:11.128196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7614404-b4ea-4f45-9d59-3c61c495fa46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:11.129628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7614404-b4ea-4f45-9d59-3c61c495fa46 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:11.136059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 408/832] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:39:10 2022] GET /identity/v3/role_assignments?user.id=3f3efa9f3eb24864943b68d241435817&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=9a439cf8eef34ada85428b378878e63e => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:11.438017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99638e-e623-49d7-8f07-261cd3c0bdb5 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:11.438529 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99638e-e623-49d7-8f07-261cd3c0bdb5 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:11.438607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99638e-e623-49d7-8f07-261cd3c0bdb5 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:11.440188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 425/833] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:11.448132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5aa804-0f5c-4c77-8efb-7a8921392718 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:11.448526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5aa804-0f5c-4c77-8efb-7a8921392718 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:11.448904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5aa804-0f5c-4c77-8efb-7a8921392718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:11.469147 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4f5aa804-0f5c-4c77-8efb-7a8921392718 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:11.470347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4f5aa804-0f5c-4c77-8efb-7a8921392718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:11.541654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 409/834] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:11.817150 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99638e-e623-49d7-8f07-261cd3c0bdb5 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:11.926019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99638e-e623-49d7-8f07-261cd3c0bdb5 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:11.935933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfe66a-02c1-455e-9513-ddce705412f2 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:11.936214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfe66a-02c1-455e-9513-ddce705412f2 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:11.936485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfe66a-02c1-455e-9513-ddce705412f2 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:11.942190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 410/835] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:39:11.943591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750cced-8e6f-4122-a8c7-7523970b34d3 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:11.943891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750cced-8e6f-4122-a8c7-7523970b34d3 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:11.944159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750cced-8e6f-4122-a8c7-7523970b34d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:11.951427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:11.952558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:11.964787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4750cced-8e6f-4122-a8c7-7523970b34d3 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:11.965258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5390c-e9fe-4690-8ae1-c59d6fee3553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:11.965897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5390c-e9fe-4690-8ae1-c59d6fee3553 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:11.966178 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5390c-e9fe-4690-8ae1-c59d6fee3553 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:11.966352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4750cced-8e6f-4122-a8c7-7523970b34d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:11.966455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5390c-e9fe-4690-8ae1-c59d6fee3553 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:11.977083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5390c-e9fe-4690-8ae1-c59d6fee3553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-9ab5390c-e9fe-4690-8ae1-c59d6fee3553 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:11.981788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:11.982922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:11.984351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 426/836] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:39:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:11.990137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5390c-e9fe-4690-8ae1-c59d6fee3553 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:12.059817 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 411/837] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:12.102805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5390c-e9fe-4690-8ae1-c59d6fee3553 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:12.138346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129b36b-f0c0-4d53-a0c4-1224e6372ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:12.138959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129b36b-f0c0-4d53-a0c4-1224e6372ef4 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:12.139239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129b36b-f0c0-4d53-a0c4-1224e6372ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:12.139516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129b36b-f0c0-4d53-a0c4-1224e6372ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:12.140786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129b36b-f0c0-4d53-a0c4-1224e6372ef4 None admin] RBAC: Adding query filter params (name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:12.141091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129b36b-f0c0-4d53-a0c4-1224e6372ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:12.142459 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129b36b-f0c0-4d53-a0c4-1224e6372ef4 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:12.151291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 427/838] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:39:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:12.160250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750cced-8e6f-4122-a8c7-7523970b34d3 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:12.265122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750cced-8e6f-4122-a8c7-7523970b34d3 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:12.301417 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fed408-3762-4b7e-aeee-77f75f0e9b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:12.302046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fed408-3762-4b7e-aeee-77f75f0e9b80 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:12.302325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fed408-3762-4b7e-aeee-77f75f0e9b80 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:12.302599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fed408-3762-4b7e-aeee-77f75f0e9b80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:12.304545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:12.314358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fed408-3762-4b7e-aeee-77f75f0e9b80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:12.315918 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fed408-3762-4b7e-aeee-77f75f0e9b80 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:12.324569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129b36b-f0c0-4d53-a0c4-1224e6372ef4 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-47fed408-3762-4b7e-aeee-77f75f0e9b80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:12.332678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:12.333628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:12.334878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:12.334878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:12.334878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:12.334878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 412/839] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:12.341855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fed408-3762-4b7e-aeee-77f75f0e9b80 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:12.383756 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:12.446834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129b36b-f0c0-4d53-a0c4-1224e6372ef4 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:12.456766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fed408-3762-4b7e-aeee-77f75f0e9b80 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:12.480152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346c579-5d4a-4ef0-98e3-98e2c96c3634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:12.480774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346c579-5d4a-4ef0-98e3-98e2c96c3634 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:12.481048 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346c579-5d4a-4ef0-98e3-98e2c96c3634 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:12.481318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346c579-5d4a-4ef0-98e3-98e2c96c3634 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:12.491525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346c579-5d4a-4ef0-98e3-98e2c96c3634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:12.494064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f29be-4d37-464c-b306-4b6d9cf87af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:12.494879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f29be-4d37-464c-b306-4b6d9cf87af2 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:12.495337 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f29be-4d37-464c-b306-4b6d9cf87af2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:12.495829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f29be-4d37-464c-b306-4b6d9cf87af2 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-f346c579-5d4a-4ef0-98e3-98e2c96c3634 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:12.496567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:12.497485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:12.498396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 428/840] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:39:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:12.499435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760abf2-3d08-4d1e-867e-02186498440f None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:12.499435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760abf2-3d08-4d1e-867e-02186498440f None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:12.500438 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760abf2-3d08-4d1e-867e-02186498440f None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:12.500689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f29be-4d37-464c-b306-4b6d9cf87af2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:12.501197 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f29be-4d37-464c-b306-4b6d9cf87af2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:12.505309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 429/841] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:12.505309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760abf2-3d08-4d1e-867e-02186498440f None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:12.505491 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f29be-4d37-464c-b306-4b6d9cf87af2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:12.519496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 413/842] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:12.520581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813565d-192c-4446-bac1-df34af8319d7 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:12.520860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813565d-192c-4446-bac1-df34af8319d7 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:12.521126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813565d-192c-4446-bac1-df34af8319d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:12.539182 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-c813565d-192c-4446-bac1-df34af8319d7 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:12.540387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c813565d-192c-4446-bac1-df34af8319d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:12.583911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:12.611777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 414/843] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:12.611777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813565d-192c-4446-bac1-df34af8319d7 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:12.626575 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760abf2-3d08-4d1e-867e-02186498440f None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:12.634281 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:12.662344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af4b9a-8b70-4045-a341-4b5a542cbfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:12.662941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af4b9a-8b70-4045-a341-4b5a542cbfa7 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:12.663222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af4b9a-8b70-4045-a341-4b5a542cbfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:12.663483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af4b9a-8b70-4045-a341-4b5a542cbfa7 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:12.664164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af4b9a-8b70-4045-a341-4b5a542cbfa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:12.664462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af4b9a-8b70-4045-a341-4b5a542cbfa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:12.665812 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af4b9a-8b70-4045-a341-4b5a542cbfa7 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:12.681522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 430/844] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:39:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:12.682855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af4b9a-8b70-4045-a341-4b5a542cbfa7 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:12.758826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813565d-192c-4446-bac1-df34af8319d7 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:12.788330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af4b9a-8b70-4045-a341-4b5a542cbfa7 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:12.810800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99463cb-926b-42ea-98bf-874d8ea18180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:12.812631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99463cb-926b-42ea-98bf-874d8ea18180 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:12.813114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99463cb-926b-42ea-98bf-874d8ea18180 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:12.813928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99463cb-926b-42ea-98bf-874d8ea18180 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:12.824385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99463cb-926b-42ea-98bf-874d8ea18180 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:12.826166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea9770-48ee-41ad-b951-bc2505680bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:12.826549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99463cb-926b-42ea-98bf-874d8ea18180 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:12.826856 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea9770-48ee-41ad-b951-bc2505680bd4 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:12.827194 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea9770-48ee-41ad-b951-bc2505680bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:12.827492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea9770-48ee-41ad-b951-bc2505680bd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:12.839366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea9770-48ee-41ad-b951-bc2505680bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:12.840829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea9770-48ee-41ad-b951-bc2505680bd4 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-f99463cb-926b-42ea-98bf-874d8ea18180 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:12.841504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:12.842450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:12.843963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:12.843963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:12.843963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:12.843963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:12.843963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:12.843963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:12.843963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:12.843963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:12.843963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 14:39:12.843963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:12.846118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 415/845] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Aug 18 14:39:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:12.850833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99463cb-926b-42ea-98bf-874d8ea18180 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-9eea9770-48ee-41ad-b951-bc2505680bd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:12.869299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:12.870254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:12.871142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:12.871142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:12.871142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:12.873576 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 431/846] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:12.881563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea9770-48ee-41ad-b951-bc2505680bd4 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:12.891838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:12.916427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:12.968884 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99463cb-926b-42ea-98bf-874d8ea18180 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:13.002178 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea9770-48ee-41ad-b951-bc2505680bd4 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:13.009835 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0fbbd-ac22-4b1f-800d-6198d552eaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:13.010996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0fbbd-ac22-4b1f-800d-6198d552eaab None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.011284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0fbbd-ac22-4b1f-800d-6198d552eaab None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.011565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0fbbd-ac22-4b1f-800d-6198d552eaab None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.012766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0fbbd-ac22-4b1f-800d-6198d552eaab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:13.013081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0fbbd-ac22-4b1f-800d-6198d552eaab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:13.015882 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0fbbd-ac22-4b1f-800d-6198d552eaab None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:13.035728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 416/847] 10.222.0.36 () {56 vars in 1149 bytes} [Thu Aug 18 14:39:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:13.035728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0fbbd-ac22-4b1f-800d-6198d552eaab None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:13.043509 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6488f-a8c4-4353-87c3-f2a3fa71cb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:13.044160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6488f-a8c4-4353-87c3-f2a3fa71cb56 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.044436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6488f-a8c4-4353-87c3-f2a3fa71cb56 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.044701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6488f-a8c4-4353-87c3-f2a3fa71cb56 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.045345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6488f-a8c4-4353-87c3-f2a3fa71cb56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:13.045641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6488f-a8c4-4353-87c3-f2a3fa71cb56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:13.047031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6488f-a8c4-4353-87c3-f2a3fa71cb56 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:13.063310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 432/848] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:13.063310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6488f-a8c4-4353-87c3-f2a3fa71cb56 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:13.137014 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:13.144730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0fbbd-ac22-4b1f-800d-6198d552eaab None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:13.146617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:13.166924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6488f-a8c4-4353-87c3-f2a3fa71cb56 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:13.180342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4be4d-cb4f-4a2f-ba9d-22a43ad10b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:13.180961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4be4d-cb4f-4a2f-ba9d-22a43ad10b40 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.181241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4be4d-cb4f-4a2f-ba9d-22a43ad10b40 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.181509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4be4d-cb4f-4a2f-ba9d-22a43ad10b40 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.182090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4be4d-cb4f-4a2f-ba9d-22a43ad10b40 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:13.183451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4be4d-cb4f-4a2f-ba9d-22a43ad10b40 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-ccb4be4d-cb4f-4a2f-ba9d-22a43ad10b40 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:13.192025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:13.193155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:13.193981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 417/849] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:39:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:13.195060 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4be4d-cb4f-4a2f-ba9d-22a43ad10b40 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:13.204278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96d15e-8c49-46b4-956b-ebe362e6e96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:13.204946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96d15e-8c49-46b4-956b-ebe362e6e96a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.205228 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96d15e-8c49-46b4-956b-ebe362e6e96a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.205500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96d15e-8c49-46b4-956b-ebe362e6e96a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.271069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96d15e-8c49-46b4-956b-ebe362e6e96a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:13.272543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96d15e-8c49-46b4-956b-ebe362e6e96a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:13.293585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4be4d-cb4f-4a2f-ba9d-22a43ad10b40 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:13.328451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f63e2-8089-42d5-ad7f-bb7acd037061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:13.329115 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f63e2-8089-42d5-ad7f-bb7acd037061 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.329386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f63e2-8089-42d5-ad7f-bb7acd037061 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.329654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f63e2-8089-42d5-ad7f-bb7acd037061 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.336986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f63e2-8089-42d5-ad7f-bb7acd037061 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:13.338415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f63e2-8089-42d5-ad7f-bb7acd037061 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:13.349383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-9f1f63e2-8089-42d5-ad7f-bb7acd037061 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:13.350249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:13.351663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 418/850] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:39:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2767b4-7c99-40b6-b089-1dbfa3a32a0d None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.352684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2767b4-7c99-40b6-b089-1dbfa3a32a0d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.353409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2767b4-7c99-40b6-b089-1dbfa3a32a0d None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.355840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 419/851] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:13.355840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2767b4-7c99-40b6-b089-1dbfa3a32a0d None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:13.373774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 433/852] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:39:13.374954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96d15e-8c49-46b4-956b-ebe362e6e96a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:13.375716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:13.457797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2767b4-7c99-40b6-b089-1dbfa3a32a0d None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:13.469970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96d15e-8c49-46b4-956b-ebe362e6e96a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:13.491700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dba418-75ce-4ffd-87ef-db0f8aff8cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:13.492310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dba418-75ce-4ffd-87ef-db0f8aff8cb0 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.492579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dba418-75ce-4ffd-87ef-db0f8aff8cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.492843 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dba418-75ce-4ffd-87ef-db0f8aff8cb0 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.493459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dba418-75ce-4ffd-87ef-db0f8aff8cb0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:13.493756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dba418-75ce-4ffd-87ef-db0f8aff8cb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:13.495107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dba418-75ce-4ffd-87ef-db0f8aff8cb0 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:13.499588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 420/853] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:39:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:13.500704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213b94d-080c-491a-9605-d9fdc29dbe7b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.501089 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213b94d-080c-491a-9605-d9fdc29dbe7b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.501364 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213b94d-080c-491a-9605-d9fdc29dbe7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.506235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fc6e6-cbdd-4b6a-81ef-d43d6b5dc520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:13.506831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fc6e6-cbdd-4b6a-81ef-d43d6b5dc520 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.507113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fc6e6-cbdd-4b6a-81ef-d43d6b5dc520 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.507384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fc6e6-cbdd-4b6a-81ef-d43d6b5dc520 None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.508144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fc6e6-cbdd-4b6a-81ef-d43d6b5dc520 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:13.508446 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fc6e6-cbdd-4b6a-81ef-d43d6b5dc520 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:13.509856 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fc6e6-cbdd-4b6a-81ef-d43d6b5dc520 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:13.518965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4213b94d-080c-491a-9605-d9fdc29dbe7b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:13.520172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4213b94d-080c-491a-9605-d9fdc29dbe7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:13.526975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 434/854] 10.222.0.36 () {56 vars in 1135 bytes} [Thu Aug 18 14:39:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:13.527792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fc6e6-cbdd-4b6a-81ef-d43d6b5dc520 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:13.568076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:13.590206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 421/855] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:13.591051 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213b94d-080c-491a-9605-d9fdc29dbe7b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:13.605322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:13.635319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fc6e6-cbdd-4b6a-81ef-d43d6b5dc520 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:13.672358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af748488-1bf8-47f1-ba62-052075ddbaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:13.672983 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af748488-1bf8-47f1-ba62-052075ddbaf3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.673684 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af748488-1bf8-47f1-ba62-052075ddbaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.673970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af748488-1bf8-47f1-ba62-052075ddbaf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.686114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af748488-1bf8-47f1-ba62-052075ddbaf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:13.688332 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af748488-1bf8-47f1-ba62-052075ddbaf3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:13.697765 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213b94d-080c-491a-9605-d9fdc29dbe7b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-af748488-1bf8-47f1-ba62-052075ddbaf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:13.703362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:13.704472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:13.710617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:13.710617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:13.710617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:13.710617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 435/856] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:13.710617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af748488-1bf8-47f1-ba62-052075ddbaf3 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:13.736679 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc2e4a-1bc8-4d02-8efe-0fd4c84caa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:13.737280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc2e4a-1bc8-4d02-8efe-0fd4c84caa15 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.737562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc2e4a-1bc8-4d02-8efe-0fd4c84caa15 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.737826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc2e4a-1bc8-4d02-8efe-0fd4c84caa15 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.738480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc2e4a-1bc8-4d02-8efe-0fd4c84caa15 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:13.738744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc2e4a-1bc8-4d02-8efe-0fd4c84caa15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:13.740115 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc2e4a-1bc8-4d02-8efe-0fd4c84caa15 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:13.744519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 422/857] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:39:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:13.745279 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc2e4a-1bc8-4d02-8efe-0fd4c84caa15 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:13.834361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:13.838582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:13.854996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af748488-1bf8-47f1-ba62-052075ddbaf3 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:13.863005 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc2e4a-1bc8-4d02-8efe-0fd4c84caa15 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:13.894332 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf674e-d700-4fa2-b805-88ddc5e626c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:13.895505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf674e-d700-4fa2-b805-88ddc5e626c3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.895505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf674e-d700-4fa2-b805-88ddc5e626c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.895505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf674e-d700-4fa2-b805-88ddc5e626c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.911808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53f7fb-ca47-43e3-a80e-baf83f5107b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:13.912420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53f7fb-ca47-43e3-a80e-baf83f5107b1 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:13.912700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53f7fb-ca47-43e3-a80e-baf83f5107b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:13.912976 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53f7fb-ca47-43e3-a80e-baf83f5107b1 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:13.913617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53f7fb-ca47-43e3-a80e-baf83f5107b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:13.914031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53f7fb-ca47-43e3-a80e-baf83f5107b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:13.915440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53f7fb-ca47-43e3-a80e-baf83f5107b1 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:13.927372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 423/858] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:13.928281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53f7fb-ca47-43e3-a80e-baf83f5107b1 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:13.973687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf674e-d700-4fa2-b805-88ddc5e626c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:13.975278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf674e-d700-4fa2-b805-88ddc5e626c3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:14.034061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53f7fb-ca47-43e3-a80e-baf83f5107b1 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:14.070591 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:14.073808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994b7c6-fd8f-4162-a548-49311c9f37a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:14.074512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994b7c6-fd8f-4162-a548-49311c9f37a7 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:14.074786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994b7c6-fd8f-4162-a548-49311c9f37a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:14.075052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994b7c6-fd8f-4162-a548-49311c9f37a7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:14.075631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994b7c6-fd8f-4162-a548-49311c9f37a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:14.077024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994b7c6-fd8f-4162-a548-49311c9f37a7 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:14.080213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:14.087931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 436/859] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:39:14.087931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf674e-d700-4fa2-b805-88ddc5e626c3 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-e994b7c6-fd8f-4162-a548-49311c9f37a7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:14.088220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:14.089258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:14.090151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 424/860] 10.222.0.36 () {56 vars in 1132 bytes} [Thu Aug 18 14:39:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:14.094816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994b7c6-fd8f-4162-a548-49311c9f37a7 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:14.189716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf674e-d700-4fa2-b805-88ddc5e626c3 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:14.200859 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994b7c6-fd8f-4162-a548-49311c9f37a7 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:14.227452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fab7e-cd82-43c7-bd3d-34ef0c703236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:14.228061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fab7e-cd82-43c7-bd3d-34ef0c703236 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:14.228330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fab7e-cd82-43c7-bd3d-34ef0c703236 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:14.228607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fab7e-cd82-43c7-bd3d-34ef0c703236 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:14.235847 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fab7e-cd82-43c7-bd3d-34ef0c703236 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:14.237245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529c854-1022-4743-8bc6-abd0f0ab7447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:14.237375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fab7e-cd82-43c7-bd3d-34ef0c703236 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:14.237843 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529c854-1022-4743-8bc6-abd0f0ab7447 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:14.238117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529c854-1022-4743-8bc6-abd0f0ab7447 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:14.238383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529c854-1022-4743-8bc6-abd0f0ab7447 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:14.239008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529c854-1022-4743-8bc6-abd0f0ab7447 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:14.239305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529c854-1022-4743-8bc6-abd0f0ab7447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:14.240674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529c854-1022-4743-8bc6-abd0f0ab7447 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:14.249817 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 425/861] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Aug 18 14:39:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:14.249817 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529c854-1022-4743-8bc6-abd0f0ab7447 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-847fab7e-cd82-43c7-bd3d-34ef0c703236 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:14.250396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:14.251614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:14.252764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:14.252764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:14.252764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:14.252764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:14.252764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:14.252764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:14.252764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:14.252764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:14.252764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 437/862] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:14.255126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fab7e-cd82-43c7-bd3d-34ef0c703236 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:14.308145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:14.315733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:14.357334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529c854-1022-4743-8bc6-abd0f0ab7447 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:14.361801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fab7e-cd82-43c7-bd3d-34ef0c703236 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:14.393258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54287b25-5966-45a1-b1b9-0e7a001c7cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:14.393912 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54287b25-5966-45a1-b1b9-0e7a001c7cc2 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:14.394443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54287b25-5966-45a1-b1b9-0e7a001c7cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:14.394443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54287b25-5966-45a1-b1b9-0e7a001c7cc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:14.397713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e87bf4-7d28-4f9b-b2ad-20b2e1f5e582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:14.398533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e87bf4-7d28-4f9b-b2ad-20b2e1f5e582 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:14.399046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e87bf4-7d28-4f9b-b2ad-20b2e1f5e582 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:14.399535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e87bf4-7d28-4f9b-b2ad-20b2e1f5e582 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:14.400492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e87bf4-7d28-4f9b-b2ad-20b2e1f5e582 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:14.400988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e87bf4-7d28-4f9b-b2ad-20b2e1f5e582 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:14.402546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e87bf4-7d28-4f9b-b2ad-20b2e1f5e582 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:14.405055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54287b25-5966-45a1-b1b9-0e7a001c7cc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:14.407001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54287b25-5966-45a1-b1b9-0e7a001c7cc2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:14.408542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 438/863] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Aug 18 14:39:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:14.409496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e87bf4-7d28-4f9b-b2ad-20b2e1f5e582 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-54287b25-5966-45a1-b1b9-0e7a001c7cc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:14.433968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:14.435015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:14.437300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:14.437300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:14.437300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:14.437300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 426/864] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:14.441248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54287b25-5966-45a1-b1b9-0e7a001c7cc2 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:14.553786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54287b25-5966-45a1-b1b9-0e7a001c7cc2 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:14.561257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e87bf4-7d28-4f9b-b2ad-20b2e1f5e582 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:14.573361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:14.582714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:14.599114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440928b5-a690-4ae3-8b6c-307c6b9a27aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:14.599412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440928b5-a690-4ae3-8b6c-307c6b9a27aa None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:14.599635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440928b5-a690-4ae3-8b6c-307c6b9a27aa None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:14.603282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440928b5-a690-4ae3-8b6c-307c6b9a27aa None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:14.603282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440928b5-a690-4ae3-8b6c-307c6b9a27aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:14.603282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440928b5-a690-4ae3-8b6c-307c6b9a27aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:14.603282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440928b5-a690-4ae3-8b6c-307c6b9a27aa None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:14.607070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a461379-1ce2-4a88-9a95-b65e912fafe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:14.607721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a461379-1ce2-4a88-9a95-b65e912fafe9 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:14.608010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a461379-1ce2-4a88-9a95-b65e912fafe9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:14.608277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a461379-1ce2-4a88-9a95-b65e912fafe9 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:14.609518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a461379-1ce2-4a88-9a95-b65e912fafe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:14.609818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a461379-1ce2-4a88-9a95-b65e912fafe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:14.611229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a461379-1ce2-4a88-9a95-b65e912fafe9 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:14.616964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 427/865] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:14.622583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440928b5-a690-4ae3-8b6c-307c6b9a27aa None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:14.625658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 439/866] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:14.632006 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a461379-1ce2-4a88-9a95-b65e912fafe9 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:14.741270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a461379-1ce2-4a88-9a95-b65e912fafe9 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:14.748012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440928b5-a690-4ae3-8b6c-307c6b9a27aa None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:14.784315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b2dba-c512-46cf-81b2-1ec46b7fefd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:14.785286 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b2dba-c512-46cf-81b2-1ec46b7fefd6 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:14.785768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b2dba-c512-46cf-81b2-1ec46b7fefd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:14.786267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b2dba-c512-46cf-81b2-1ec46b7fefd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:14.787203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b2dba-c512-46cf-81b2-1ec46b7fefd6 None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=cbb13837e1694d7aa928ac855e21d1a9) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:14.787736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b2dba-c512-46cf-81b2-1ec46b7fefd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:14.787998 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715a3dc-e2b1-40bc-8592-98e3e4e33926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:14.788621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715a3dc-e2b1-40bc-8592-98e3e4e33926 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:14.788894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715a3dc-e2b1-40bc-8592-98e3e4e33926 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:14.789162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715a3dc-e2b1-40bc-8592-98e3e4e33926 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:14.790210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b2dba-c512-46cf-81b2-1ec46b7fefd6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:14.796759 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715a3dc-e2b1-40bc-8592-98e3e4e33926 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:14.798193 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715a3dc-e2b1-40bc-8592-98e3e4e33926 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:14.803953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 440/867] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:39:14 2022] GET /identity/v3/role_assignments?user.id=cbb13837e1694d7aa928ac855e21d1a9&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-c715a3dc-e2b1-40bc-8592-98e3e4e33926 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:14.815218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:14.816231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:14.817146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:14.817146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:14.817146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:14.817146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:14.817146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:14.817146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:14.817146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:14.817146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:14.817146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 428/868] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:14.824388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b2dba-c512-46cf-81b2-1ec46b7fefd6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:14.870174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:14.951329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b2dba-c512-46cf-81b2-1ec46b7fefd6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:14.989693 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be40d9b-466b-46c2-b4cc-ef4db038636d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:14.990559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be40d9b-466b-46c2-b4cc-ef4db038636d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:14.990838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be40d9b-466b-46c2-b4cc-ef4db038636d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:14.991116 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be40d9b-466b-46c2-b4cc-ef4db038636d None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:14.991838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be40d9b-466b-46c2-b4cc-ef4db038636d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:14.992145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be40d9b-466b-46c2-b4cc-ef4db038636d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:14.993569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be40d9b-466b-46c2-b4cc-ef4db038636d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:15.006478 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 441/869] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:15.012444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715a3dc-e2b1-40bc-8592-98e3e4e33926 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:15.049874 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:15.118309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715a3dc-e2b1-40bc-8592-98e3e4e33926 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:15.154490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472120b-3eae-4605-a610-f646a914fe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:15.155089 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472120b-3eae-4605-a610-f646a914fe8f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:15.155369 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472120b-3eae-4605-a610-f646a914fe8f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:15.156217 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472120b-3eae-4605-a610-f646a914fe8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:15.156940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472120b-3eae-4605-a610-f646a914fe8f None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=f578727b96564c3db6b5a1e6cac109d8) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:15.157246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472120b-3eae-4605-a610-f646a914fe8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:15.158676 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472120b-3eae-4605-a610-f646a914fe8f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:15.165069 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 429/870] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:39:15 2022] GET /identity/v3/role_assignments?user.id=f578727b96564c3db6b5a1e6cac109d8&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=9a439cf8eef34ada85428b378878e63e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:15.677480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa346786-0020-4216-88fa-efeb7a9c00ce None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:15.678027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa346786-0020-4216-88fa-efeb7a9c00ce None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:15.678027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa346786-0020-4216-88fa-efeb7a9c00ce None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:15.679572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 442/871] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:15.684889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1673f2e-7744-4c42-bb7e-d7e8e268dfcf None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:15.685277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1673f2e-7744-4c42-bb7e-d7e8e268dfcf None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:15.685440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1673f2e-7744-4c42-bb7e-d7e8e268dfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:15.704105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-f1673f2e-7744-4c42-bb7e-d7e8e268dfcf None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:15.705302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f1673f2e-7744-4c42-bb7e-d7e8e268dfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:15.774212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 430/872] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:16.040511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa346786-0020-4216-88fa-efeb7a9c00ce None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:16.139336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:16.144084 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa346786-0020-4216-88fa-efeb7a9c00ce None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:16.178919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4156ea-a518-416e-9f04-5d36ce4eb623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:16.179527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4156ea-a518-416e-9f04-5d36ce4eb623 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:16.179837 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4156ea-a518-416e-9f04-5d36ce4eb623 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:16.180170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4156ea-a518-416e-9f04-5d36ce4eb623 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:16.180828 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4156ea-a518-416e-9f04-5d36ce4eb623 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:16.182192 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4156ea-a518-416e-9f04-5d36ce4eb623 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:16.204009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 443/873] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:39:16 2022] POST /identity/v3/services => generated 251 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:16.354604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a0c28-f49c-4f0a-ae00-89e511b24426 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:16.354956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a0c28-f49c-4f0a-ae00-89e511b24426 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:16.355129 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a0c28-f49c-4f0a-ae00-89e511b24426 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:16.356685 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 431/874] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:16.362430 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032bf22-a957-4565-a901-d0b520aca8fc None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:16.362723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032bf22-a957-4565-a901-d0b520aca8fc None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:16.363001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032bf22-a957-4565-a901-d0b520aca8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:16.381754 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f032bf22-a957-4565-a901-d0b520aca8fc None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:16.382965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f032bf22-a957-4565-a901-d0b520aca8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:16.462552 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 444/875] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:16.726497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a0c28-f49c-4f0a-ae00-89e511b24426 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:16.758772 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:16.833473 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a0c28-f49c-4f0a-ae00-89e511b24426 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:16.868900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d87942-90fd-4b76-802c-ed570ce4c968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:16.869508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d87942-90fd-4b76-802c-ed570ce4c968 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:16.869785 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d87942-90fd-4b76-802c-ed570ce4c968 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:16.870057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d87942-90fd-4b76-802c-ed570ce4c968 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:16.870709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d87942-90fd-4b76-802c-ed570ce4c968 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:16.872110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d87942-90fd-4b76-802c-ed570ce4c968 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:16.890509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 432/876] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:39:16 2022] POST /identity/v3/services => generated 269 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:16.926370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9756e6-772a-40f0-b485-550831ce85e0 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:16.926933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9756e6-772a-40f0-b485-550831ce85e0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:16.926933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9756e6-772a-40f0-b485-550831ce85e0 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:16.928744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 445/877] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:16.934535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba8116-9983-47aa-88f9-e3cba3a27342 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:16.934819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba8116-9983-47aa-88f9-e3cba3a27342 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:16.935319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba8116-9983-47aa-88f9-e3cba3a27342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:16.956262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-f8ba8116-9983-47aa-88f9-e3cba3a27342 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:16.957884 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f8ba8116-9983-47aa-88f9-e3cba3a27342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:17.040105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 433/878] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:17.159939 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b213e-f1fb-46b7-81e7-9bef83782a83 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:17.160595 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b213e-f1fb-46b7-81e7-9bef83782a83 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:17.161090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b213e-f1fb-46b7-81e7-9bef83782a83 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:17.162858 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 446/879] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:17.168196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817db90d-ed8f-4e6d-ba2e-6269c6640ea8 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:17.168744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817db90d-ed8f-4e6d-ba2e-6269c6640ea8 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:17.169221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817db90d-ed8f-4e6d-ba2e-6269c6640ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:17.187833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-817db90d-ed8f-4e6d-ba2e-6269c6640ea8 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:17.189312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-817db90d-ed8f-4e6d-ba2e-6269c6640ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:17.200422 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:17.264406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 434/880] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:17.313682 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b213e-f1fb-46b7-81e7-9bef83782a83 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:17.410915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:17.429079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b213e-f1fb-46b7-81e7-9bef83782a83 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:17.465506 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0d5de-c94a-4546-b044-890c449394ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:17.466135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0d5de-c94a-4546-b044-890c449394ef None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:17.466408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0d5de-c94a-4546-b044-890c449394ef None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:17.466675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0d5de-c94a-4546-b044-890c449394ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:17.477094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0d5de-c94a-4546-b044-890c449394ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-90f0d5de-c94a-4546-b044-890c449394ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:17.481738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:17.482669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:17.483960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 447/881] 10.222.0.36 () {56 vars in 1120 bytes} [Thu Aug 18 14:39:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:17.490721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817db90d-ed8f-4e6d-ba2e-6269c6640ea8 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:17.541413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0d5de-c94a-4546-b044-890c449394ef None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:17.592336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817db90d-ed8f-4e6d-ba2e-6269c6640ea8 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:17.628055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bc7d3-9022-4bde-8147-167e7d67a2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:17.628656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bc7d3-9022-4bde-8147-167e7d67a2b7 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:17.630206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bc7d3-9022-4bde-8147-167e7d67a2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:17.630480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bc7d3-9022-4bde-8147-167e7d67a2b7 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:17.631143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bc7d3-9022-4bde-8147-167e7d67a2b7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:17.631444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bc7d3-9022-4bde-8147-167e7d67a2b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:17.633812 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bc7d3-9022-4bde-8147-167e7d67a2b7 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:17.648889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 435/882] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Aug 18 14:39:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:17.651092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bc7d3-9022-4bde-8147-167e7d67a2b7 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:17.665734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:17.671912 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0d5de-c94a-4546-b044-890c449394ef None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:17.714969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf7793-a218-43da-8ba3-d4c8a5608a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:17.715775 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf7793-a218-43da-8ba3-d4c8a5608a76 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:17.716312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf7793-a218-43da-8ba3-d4c8a5608a76 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:17.716683 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf7793-a218-43da-8ba3-d4c8a5608a76 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:17.717651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf7793-a218-43da-8ba3-d4c8a5608a76 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:17.719137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf7793-a218-43da-8ba3-d4c8a5608a76 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-5dcf7793-a218-43da-8ba3-d4c8a5608a76 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:17.730492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:17.731728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:17.732759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:17.735834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 448/883] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Aug 18 14:39:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:17.743661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf7793-a218-43da-8ba3-d4c8a5608a76 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:17.758049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:17.786061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bc7d3-9022-4bde-8147-167e7d67a2b7 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:17.823960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0aa60-63ad-4ba9-a8f4-e1da531fb571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:17.824939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0aa60-63ad-4ba9-a8f4-e1da531fb571 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:17.825421 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0aa60-63ad-4ba9-a8f4-e1da531fb571 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:17.825967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0aa60-63ad-4ba9-a8f4-e1da531fb571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:17.837620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0aa60-63ad-4ba9-a8f4-e1da531fb571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:17.839418 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0aa60-63ad-4ba9-a8f4-e1da531fb571 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-10b0aa60-63ad-4ba9-a8f4-e1da531fb571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:17.854592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:17.855559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:17.856482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:17.856482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:17.856482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:17.859140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 436/884] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:17.863713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf7793-a218-43da-8ba3-d4c8a5608a76 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:17.865566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0aa60-63ad-4ba9-a8f4-e1da531fb571 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:17.899062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d96e49-5289-4cb8-a513-a7953c9d3f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:17.899062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d96e49-5289-4cb8-a513-a7953c9d3f7d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:17.899062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d96e49-5289-4cb8-a513-a7953c9d3f7d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:17.899062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d96e49-5289-4cb8-a513-a7953c9d3f7d None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:17.899614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d96e49-5289-4cb8-a513-a7953c9d3f7d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:17.899614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d96e49-5289-4cb8-a513-a7953c9d3f7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:17.899781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d96e49-5289-4cb8-a513-a7953c9d3f7d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:17.909359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 449/885] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Aug 18 14:39:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:17.910738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d96e49-5289-4cb8-a513-a7953c9d3f7d None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:17.942074 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:17.984560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:17.987396 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0aa60-63ad-4ba9-a8f4-e1da531fb571 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:18.017320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d96e49-5289-4cb8-a513-a7953c9d3f7d None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:18.024975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f18212-55fa-4796-a64e-0daadb2d95dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:18.025849 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f18212-55fa-4796-a64e-0daadb2d95dd None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:18.026276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f18212-55fa-4796-a64e-0daadb2d95dd None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:18.026727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f18212-55fa-4796-a64e-0daadb2d95dd None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:18.027545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18212-55fa-4796-a64e-0daadb2d95dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:18.028022 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18212-55fa-4796-a64e-0daadb2d95dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:18.029602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18212-55fa-4796-a64e-0daadb2d95dd None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:18.048008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 437/886] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:18.050042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f18212-55fa-4796-a64e-0daadb2d95dd None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:18.054951 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da9275-b252-4dd2-8f17-4ddb97462f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:18.055417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da9275-b252-4dd2-8f17-4ddb97462f20 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:18.055707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da9275-b252-4dd2-8f17-4ddb97462f20 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:18.055982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da9275-b252-4dd2-8f17-4ddb97462f20 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:18.056608 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da9275-b252-4dd2-8f17-4ddb97462f20 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:18.056909 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da9275-b252-4dd2-8f17-4ddb97462f20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:18.058273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da9275-b252-4dd2-8f17-4ddb97462f20 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:18.071837 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 450/887] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Aug 18 14:39:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:18.170247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f18212-55fa-4796-a64e-0daadb2d95dd None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:18.193334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:18.209170 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667db85c-16e2-49e1-a0bb-469ed289810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:18.209852 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667db85c-16e2-49e1-a0bb-469ed289810e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:18.210191 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667db85c-16e2-49e1-a0bb-469ed289810e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:18.210515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667db85c-16e2-49e1-a0bb-469ed289810e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:18.218576 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667db85c-16e2-49e1-a0bb-469ed289810e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:18.220119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667db85c-16e2-49e1-a0bb-469ed289810e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-667db85c-16e2-49e1-a0bb-469ed289810e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:18.231915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:18.232862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:18.234405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:18.234405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:18.234405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:18.234405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:18.234405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:18.234405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:18.234405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:18.234405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:18.234405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 14:39:18.234405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:18.234405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 438/888] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Aug 18 14:39:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:18.242527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da9275-b252-4dd2-8f17-4ddb97462f20 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:18.249974 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0184-3cc3-4a6c-ad1f-d5b510fe4854 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:18.250252 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0184-3cc3-4a6c-ad1f-d5b510fe4854 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:18.250526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0184-3cc3-4a6c-ad1f-d5b510fe4854 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:18.252128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 439/889] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:39:18.257111 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614334-5e1b-48d6-8e7f-f4709d4f6889 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:18.257387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614334-5e1b-48d6-8e7f-f4709d4f6889 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:18.257654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614334-5e1b-48d6-8e7f-f4709d4f6889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:18.275201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-9b614334-5e1b-48d6-8e7f-f4709d4f6889 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:18.276417 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9b614334-5e1b-48d6-8e7f-f4709d4f6889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:18.349710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 440/890] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:18.368005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da9275-b252-4dd2-8f17-4ddb97462f20 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:18.403397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708507ac-63f9-4fea-b265-8b8037b5b181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:18.404320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708507ac-63f9-4fea-b265-8b8037b5b181 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:18.404776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708507ac-63f9-4fea-b265-8b8037b5b181 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:18.405217 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708507ac-63f9-4fea-b265-8b8037b5b181 None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:18.406744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708507ac-63f9-4fea-b265-8b8037b5b181 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:18.407224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708507ac-63f9-4fea-b265-8b8037b5b181 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:18.408845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708507ac-63f9-4fea-b265-8b8037b5b181 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:18.411764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:18.428668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 451/891] 10.222.0.36 () {56 vars in 1149 bytes} [Thu Aug 18 14:39:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:18.434189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614334-5e1b-48d6-8e7f-f4709d4f6889 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:18.540391 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614334-5e1b-48d6-8e7f-f4709d4f6889 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:18.552341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:18.575506 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8588574-e5d8-4a69-b6a0-3df4b163797c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:18.576143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8588574-e5d8-4a69-b6a0-3df4b163797c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:18.576415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8588574-e5d8-4a69-b6a0-3df4b163797c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:18.576686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8588574-e5d8-4a69-b6a0-3df4b163797c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:18.618847 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708507ac-63f9-4fea-b265-8b8037b5b181 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:18.647569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8588574-e5d8-4a69-b6a0-3df4b163797c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:18.649102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8588574-e5d8-4a69-b6a0-3df4b163797c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:18.742424 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708507ac-63f9-4fea-b265-8b8037b5b181 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:18.760720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 441/892] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:18 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:39:18.766681 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8588574-e5d8-4a69-b6a0-3df4b163797c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:18.778405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:18.788012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10c12d-ae49-4f01-a5d6-09c83a517489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:18.788649 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10c12d-ae49-4f01-a5d6-09c83a517489 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:18.788924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10c12d-ae49-4f01-a5d6-09c83a517489 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:18.789190 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10c12d-ae49-4f01-a5d6-09c83a517489 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:18.789771 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10c12d-ae49-4f01-a5d6-09c83a517489 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:18.791127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10c12d-ae49-4f01-a5d6-09c83a517489 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-1a10c12d-ae49-4f01-a5d6-09c83a517489 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:18.802628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:18.803636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:18.804530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 452/893] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:39:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:18.812453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10c12d-ae49-4f01-a5d6-09c83a517489 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:18.875233 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:18.880611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8588574-e5d8-4a69-b6a0-3df4b163797c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:18.921944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97a7db-fd92-40ad-a245-c505f4db0f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:18.922722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97a7db-fd92-40ad-a245-c505f4db0f7c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:18.923132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97a7db-fd92-40ad-a245-c505f4db0f7c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:18.923520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97a7db-fd92-40ad-a245-c505f4db0f7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:18.924224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10c12d-ae49-4f01-a5d6-09c83a517489 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:18.936402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97a7db-fd92-40ad-a245-c505f4db0f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:18.938536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97a7db-fd92-40ad-a245-c505f4db0f7c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-6b97a7db-fd92-40ad-a245-c505f4db0f7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:18.954315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:18.955276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:18.956205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:18.956205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:18.956205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:18.956205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 442/894] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:18.957027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65706d1-8ff1-4501-a81b-824f23adc405 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:18.957166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65706d1-8ff1-4501-a81b-824f23adc405 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:18.957638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65706d1-8ff1-4501-a81b-824f23adc405 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:18.959124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 443/895] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:18.963649 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65706d1-8ff1-4501-a81b-824f23adc405 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:18.965151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b76e9c-4880-4f11-9942-d07e32c16a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:18.965990 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b76e9c-4880-4f11-9942-d07e32c16a58 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:18.966714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b76e9c-4880-4f11-9942-d07e32c16a58 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:18.967999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b76e9c-4880-4f11-9942-d07e32c16a58 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:18.968653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b76e9c-4880-4f11-9942-d07e32c16a58 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:18.968959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b76e9c-4880-4f11-9942-d07e32c16a58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:18.970314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b76e9c-4880-4f11-9942-d07e32c16a58 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:18.975310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 453/896] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:39:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:18.980798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b19f7-d788-4310-9ef7-af3149a06435 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:18.981081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b19f7-d788-4310-9ef7-af3149a06435 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:18.981350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b19f7-d788-4310-9ef7-af3149a06435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:18.999867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-105b19f7-d788-4310-9ef7-af3149a06435 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:19.001005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-105b19f7-d788-4310-9ef7-af3149a06435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:19.028487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:19.070652 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65706d1-8ff1-4501-a81b-824f23adc405 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:19.071987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 454/897] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:19.081020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b19f7-d788-4310-9ef7-af3149a06435 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:19.107108 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23772ab-216e-49e6-b984-a75c9e89d027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:19.107913 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23772ab-216e-49e6-b984-a75c9e89d027 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:19.108293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23772ab-216e-49e6-b984-a75c9e89d027 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:19.108727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23772ab-216e-49e6-b984-a75c9e89d027 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:19.109468 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23772ab-216e-49e6-b984-a75c9e89d027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:19.110289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23772ab-216e-49e6-b984-a75c9e89d027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:19.111397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23772ab-216e-49e6-b984-a75c9e89d027 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:19.129305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 444/898] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:19.130753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23772ab-216e-49e6-b984-a75c9e89d027 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:19.155941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:19.207693 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b19f7-d788-4310-9ef7-af3149a06435 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:19.243141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78fcc9-0939-46bd-ab26-286624890676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:19.243814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78fcc9-0939-46bd-ab26-286624890676 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:19.244093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78fcc9-0939-46bd-ab26-286624890676 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:19.244367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78fcc9-0939-46bd-ab26-286624890676 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:19.244998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78fcc9-0939-46bd-ab26-286624890676 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:19.245300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78fcc9-0939-46bd-ab26-286624890676 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:19.246719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78fcc9-0939-46bd-ab26-286624890676 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:19.257157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 455/899] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:39:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:19.258098 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78fcc9-0939-46bd-ab26-286624890676 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:19.264773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23772ab-216e-49e6-b984-a75c9e89d027 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:19.288075 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:19.301273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48aa79e-0681-4ab8-b245-8762948fafba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:19.302101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48aa79e-0681-4ab8-b245-8762948fafba None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:19.302522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48aa79e-0681-4ab8-b245-8762948fafba None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:19.302937 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48aa79e-0681-4ab8-b245-8762948fafba None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:19.309784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48aa79e-0681-4ab8-b245-8762948fafba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:19.311461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48aa79e-0681-4ab8-b245-8762948fafba None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-b48aa79e-0681-4ab8-b245-8762948fafba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:19.323309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:19.324387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:19.325452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:19.325452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:19.325452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:19.325452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:19.325452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:19.325452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:19.325452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:19.325452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:19.332194 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 445/900] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:19.333300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48aa79e-0681-4ab8-b245-8762948fafba None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:19.375157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78fcc9-0939-46bd-ab26-286624890676 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:19.390999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:19.408961 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48063e-afc6-4097-b4ed-365b906e00af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:19.409567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48063e-afc6-4097-b4ed-365b906e00af None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:19.409847 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48063e-afc6-4097-b4ed-365b906e00af None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:19.410122 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48063e-afc6-4097-b4ed-365b906e00af None admin] PATH_INFO: `/v3/endpoints` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:19.410822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48063e-afc6-4097-b4ed-365b906e00af None admin] RBAC: Adding query filter params (interface=public, service_id=37a381b8dadb4e709ea11d919230e5c7, region_id=RegionOne) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:19.411130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48063e-afc6-4097-b4ed-365b906e00af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:19.412532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48063e-afc6-4097-b4ed-365b906e00af None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:19.420534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 456/901] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Aug 18 14:39:19 2022] GET /identity/v3/endpoints?service_id=37a381b8dadb4e709ea11d919230e5c7&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:19.421887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48063e-afc6-4097-b4ed-365b906e00af None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:19.450754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48aa79e-0681-4ab8-b245-8762948fafba None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:19.486375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46660441-1030-44af-bc22-d29fb46b95c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:19.487349 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46660441-1030-44af-bc22-d29fb46b95c7 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:19.487921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46660441-1030-44af-bc22-d29fb46b95c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:19.488243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46660441-1030-44af-bc22-d29fb46b95c7 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:19.488928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46660441-1030-44af-bc22-d29fb46b95c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:19.489227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46660441-1030-44af-bc22-d29fb46b95c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:19.490761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46660441-1030-44af-bc22-d29fb46b95c7 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:19.510125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 446/902] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:19.511039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46660441-1030-44af-bc22-d29fb46b95c7 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:19.521008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48063e-afc6-4097-b4ed-365b906e00af None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:19.556667 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39e759-37f1-47b6-ab33-5b7532d4fc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:19.557294 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39e759-37f1-47b6-ab33-5b7532d4fc2f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:19.557582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39e759-37f1-47b6-ab33-5b7532d4fc2f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:19.557859 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:19.557983 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39e759-37f1-47b6-ab33-5b7532d4fc2f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:19.558451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39e759-37f1-47b6-ab33-5b7532d4fc2f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:19.559822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39e759-37f1-47b6-ab33-5b7532d4fc2f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-9c39e759-37f1-47b6-ab33-5b7532d4fc2f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:19.569372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:19.570342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:19.571265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 457/903] 10.222.0.36 () {56 vars in 1132 bytes} [Thu Aug 18 14:39:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:19.572106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39e759-37f1-47b6-ab33-5b7532d4fc2f None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:19.620875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:19.622813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46660441-1030-44af-bc22-d29fb46b95c7 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:19.659176 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a8c9e-d0fb-4d7d-b0fc-8b9b55720643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:19.659860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a8c9e-d0fb-4d7d-b0fc-8b9b55720643 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:19.660257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a8c9e-d0fb-4d7d-b0fc-8b9b55720643 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:19.660526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a8c9e-d0fb-4d7d-b0fc-8b9b55720643 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:19.661241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a8c9e-d0fb-4d7d-b0fc-8b9b55720643 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:19.662702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a8c9e-d0fb-4d7d-b0fc-8b9b55720643 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:19.679713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 447/904] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Aug 18 14:39:19 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:39:19.679713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a8c9e-d0fb-4d7d-b0fc-8b9b55720643 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:19.693719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39e759-37f1-47b6-ab33-5b7532d4fc2f None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:19.734436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9db22-f080-497d-bf3e-2f67399277eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:19.735144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9db22-f080-497d-bf3e-2f67399277eb None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:19.735482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9db22-f080-497d-bf3e-2f67399277eb None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:19.735860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9db22-f080-497d-bf3e-2f67399277eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:19.736758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9db22-f080-497d-bf3e-2f67399277eb None admin] RBAC: Adding query filter params (role.id=b5a4a4e7850147889e8dfd1344c75440, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=cbb13837e1694d7aa928ac855e21d1a9) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:19.737129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9db22-f080-497d-bf3e-2f67399277eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:19.738781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9db22-f080-497d-bf3e-2f67399277eb None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:19.752186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 458/905] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:39:19 2022] GET /identity/v3/role_assignments?user.id=cbb13837e1694d7aa928ac855e21d1a9&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=b5a4a4e7850147889e8dfd1344c75440 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:19.786695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a8c9e-d0fb-4d7d-b0fc-8b9b55720643 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:19.795291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:19.828194 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a2f98-65f1-4afa-9fef-f55c69790ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:19.828829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a2f98-65f1-4afa-9fef-f55c69790ff6 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:19.829107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a2f98-65f1-4afa-9fef-f55c69790ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:19.829383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a2f98-65f1-4afa-9fef-f55c69790ff6 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:19.830033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a2f98-65f1-4afa-9fef-f55c69790ff6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:19.830340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a2f98-65f1-4afa-9fef-f55c69790ff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:19.833143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a2f98-65f1-4afa-9fef-f55c69790ff6 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:19.837793 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 448/906] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Aug 18 14:39:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:19.844334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9db22-f080-497d-bf3e-2f67399277eb None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:19.965137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9db22-f080-497d-bf3e-2f67399277eb None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:19.975497 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:20.000601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa940b-6714-4926-b5b6-9ba0b0fe2cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:20.002017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa940b-6714-4926-b5b6-9ba0b0fe2cb9 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:20.002496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa940b-6714-4926-b5b6-9ba0b0fe2cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:20.002963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa940b-6714-4926-b5b6-9ba0b0fe2cb9 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:20.003876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa940b-6714-4926-b5b6-9ba0b0fe2cb9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:20.004389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa940b-6714-4926-b5b6-9ba0b0fe2cb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:20.005948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa940b-6714-4926-b5b6-9ba0b0fe2cb9 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:20.011602 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 459/907] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Aug 18 14:39:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:20.018029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731119a6-abae-4b11-9d82-1e9ac68505ae None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:20.018526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa940b-6714-4926-b5b6-9ba0b0fe2cb9 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:20.018618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731119a6-abae-4b11-9d82-1e9ac68505ae None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:20.018618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731119a6-abae-4b11-9d82-1e9ac68505ae None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:20.020299 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 449/908] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:20.028217 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dc93f-7538-4d13-bcf4-3bfc2fad3e24 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:20.028503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dc93f-7538-4d13-bcf4-3bfc2fad3e24 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:20.028779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dc93f-7538-4d13-bcf4-3bfc2fad3e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:20.048767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-834dc93f-7538-4d13-bcf4-3bfc2fad3e24 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:20.050020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-834dc93f-7538-4d13-bcf4-3bfc2fad3e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:20.124843 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 450/909] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:20.125440 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa940b-6714-4926-b5b6-9ba0b0fe2cb9 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:20.166793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6f50c-dfc2-4918-8110-c9e23d21e9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:20.167846 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6f50c-dfc2-4918-8110-c9e23d21e9f8 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:20.168328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6f50c-dfc2-4918-8110-c9e23d21e9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:20.168783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6f50c-dfc2-4918-8110-c9e23d21e9f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:20.169650 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6f50c-dfc2-4918-8110-c9e23d21e9f8 None admin] RBAC: Adding query filter params (interface=public, service_id=b5af54f3c29244d0ac59b60bfe5e62b0, region_id=RegionOne) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:20.170180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6f50c-dfc2-4918-8110-c9e23d21e9f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:20.171755 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6f50c-dfc2-4918-8110-c9e23d21e9f8 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:20.177829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 460/910] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Aug 18 14:39:20 2022] GET /identity/v3/endpoints?service_id=b5af54f3c29244d0ac59b60bfe5e62b0&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:20.183532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dc93f-7538-4d13-bcf4-3bfc2fad3e24 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:20.287114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:20.291049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dc93f-7538-4d13-bcf4-3bfc2fad3e24 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:20.326068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc586113-8b1d-4afd-8ffa-4d952a6ca4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:20.326972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc586113-8b1d-4afd-8ffa-4d952a6ca4b6 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:20.327997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc586113-8b1d-4afd-8ffa-4d952a6ca4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:20.328456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc586113-8b1d-4afd-8ffa-4d952a6ca4b6 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:20.329313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc586113-8b1d-4afd-8ffa-4d952a6ca4b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:20.330842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc586113-8b1d-4afd-8ffa-4d952a6ca4b6 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:20.337009 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 451/911] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Aug 18 14:39:20 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:39:20.398018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6f50c-dfc2-4918-8110-c9e23d21e9f8 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:20.438796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:20.509981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6f50c-dfc2-4918-8110-c9e23d21e9f8 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:20.544570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e669a82-4d49-4d1c-b46c-4bc477155995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:20.545186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e669a82-4d49-4d1c-b46c-4bc477155995 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:20.545467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e669a82-4d49-4d1c-b46c-4bc477155995 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:20.545742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e669a82-4d49-4d1c-b46c-4bc477155995 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:20.546425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e669a82-4d49-4d1c-b46c-4bc477155995 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:20.547811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e669a82-4d49-4d1c-b46c-4bc477155995 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:20.565702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 461/912] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:39:20 2022] POST /identity/v3/services => generated 241 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:21.664714 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1327c-8d60-4760-be2f-c899c06e1f85 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:21.665237 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1327c-8d60-4760-be2f-c899c06e1f85 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:21.665237 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1327c-8d60-4760-be2f-c899c06e1f85 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:21.666892 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 452/913] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:21.673098 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a9a3a-1d2e-4992-86a2-3a6c84cafca0 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:21.673616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a9a3a-1d2e-4992-86a2-3a6c84cafca0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:21.673616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a9a3a-1d2e-4992-86a2-3a6c84cafca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:21.692560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-4c6a9a3a-1d2e-4992-86a2-3a6c84cafca0 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:21.693750 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4c6a9a3a-1d2e-4992-86a2-3a6c84cafca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:21.714635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c596eb0-00c5-4c05-8f19-fc08571df041 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:21.714928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c596eb0-00c5-4c05-8f19-fc08571df041 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:21.715177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c596eb0-00c5-4c05-8f19-fc08571df041 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:21.716659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 453/914] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:39:21.721618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a684c87-4ae1-4f06-a15c-3540be6706e9 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:21.721902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a684c87-4ae1-4f06-a15c-3540be6706e9 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:21.722182 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a684c87-4ae1-4f06-a15c-3540be6706e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:21.742605 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3a684c87-4ae1-4f06-a15c-3540be6706e9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:21.743953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3a684c87-4ae1-4f06-a15c-3540be6706e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:21.780768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 462/915] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:21.823179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 454/916] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:22.067268 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a9a3a-1d2e-4992-86a2-3a6c84cafca0 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:22.092299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:22.104554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a684c87-4ae1-4f06-a15c-3540be6706e9 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:22.192109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a9a3a-1d2e-4992-86a2-3a6c84cafca0 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:22.218897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:22.227387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a684c87-4ae1-4f06-a15c-3540be6706e9 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:22.239600 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe692f-1926-4e0c-8921-248f745804e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:22.240276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe692f-1926-4e0c-8921-248f745804e9 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:22.240554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe692f-1926-4e0c-8921-248f745804e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:22.240825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe692f-1926-4e0c-8921-248f745804e9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:22.241420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe692f-1926-4e0c-8921-248f745804e9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:22.242909 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe692f-1926-4e0c-8921-248f745804e9 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-6dbe692f-1926-4e0c-8921-248f745804e9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:22.252387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:22.253417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 14:39:22.254253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:22.258420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 463/917] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:39:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:22.259540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe692f-1926-4e0c-8921-248f745804e9 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:22.279095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be741a-f33b-43a7-97f8-cf9231134968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:22.280268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be741a-f33b-43a7-97f8-cf9231134968 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:22.281256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be741a-f33b-43a7-97f8-cf9231134968 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:22.282069 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be741a-f33b-43a7-97f8-cf9231134968 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:22.296858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be741a-f33b-43a7-97f8-cf9231134968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-f3be741a-f33b-43a7-97f8-cf9231134968 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:22.302981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:22.304157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:22.305192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:22.305192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:22.305192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:22.305192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:22.305192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:22.305192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:22.305192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:22.305192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:22.305192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:22.305192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:22.305192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 14:39:22.305192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:22.312622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 455/918] 10.222.0.36 () {56 vars in 1120 bytes} [Thu Aug 18 14:39:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:22.315635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be741a-f33b-43a7-97f8-cf9231134968 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:22.359281 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:22.376487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe692f-1926-4e0c-8921-248f745804e9 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:22.411694 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8754d-359c-415a-877c-27347cf749a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:22.412301 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8754d-359c-415a-877c-27347cf749a0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:22.412576 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8754d-359c-415a-877c-27347cf749a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:22.412845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8754d-359c-415a-877c-27347cf749a0 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:22.413468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8754d-359c-415a-877c-27347cf749a0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:22.413765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8754d-359c-415a-877c-27347cf749a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:22.415122 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8754d-359c-415a-877c-27347cf749a0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:22.419831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 464/919] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:39:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:22.420995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8afbfc-2a0e-4c0d-b85c-5da154745993 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:22.421272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8afbfc-2a0e-4c0d-b85c-5da154745993 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:22.421539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8afbfc-2a0e-4c0d-b85c-5da154745993 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:22.422868 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 465/920] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:22.427601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8afbfc-2a0e-4c0d-b85c-5da154745993 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:22.432929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be741a-f33b-43a7-97f8-cf9231134968 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:22.472400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836580b3-bed5-4424-901d-b2ee072418c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:22.473096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836580b3-bed5-4424-901d-b2ee072418c0 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:22.473431 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836580b3-bed5-4424-901d-b2ee072418c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:22.473759 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836580b3-bed5-4424-901d-b2ee072418c0 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:22.474576 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836580b3-bed5-4424-901d-b2ee072418c0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:22.474980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836580b3-bed5-4424-901d-b2ee072418c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:22.476581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836580b3-bed5-4424-901d-b2ee072418c0 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:22.485764 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 456/921] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Aug 18 14:39:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:22.486962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824908f-a0cd-46da-bb2a-ab7dbfbcb0b2 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:22.487242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824908f-a0cd-46da-bb2a-ab7dbfbcb0b2 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:22.487515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824908f-a0cd-46da-bb2a-ab7dbfbcb0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:22.506473 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-0824908f-a0cd-46da-bb2a-ab7dbfbcb0b2 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:22.507705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0824908f-a0cd-46da-bb2a-ab7dbfbcb0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:22.518801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:22.540716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8afbfc-2a0e-4c0d-b85c-5da154745993 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:22.574411 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db75f0f-db0d-42c9-bdf3-c537f884a4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:22.575005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db75f0f-db0d-42c9-bdf3-c537f884a4b5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:22.575289 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db75f0f-db0d-42c9-bdf3-c537f884a4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:22.575437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 457/922] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:22.575604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db75f0f-db0d-42c9-bdf3-c537f884a4b5 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:22.576198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824908f-a0cd-46da-bb2a-ab7dbfbcb0b2 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:22.576299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db75f0f-db0d-42c9-bdf3-c537f884a4b5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:22.576625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db75f0f-db0d-42c9-bdf3-c537f884a4b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:22.577941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db75f0f-db0d-42c9-bdf3-c537f884a4b5 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:22.582747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 466/923] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Aug 18 14:39:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:22.583966 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4e33a-2364-465b-9c82-204ead3c43c1 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:22.584250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4e33a-2364-465b-9c82-204ead3c43c1 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:22.584523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4e33a-2364-465b-9c82-204ead3c43c1 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:22.585695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 467/924] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:22.590550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4e33a-2364-465b-9c82-204ead3c43c1 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:22.606272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:22.679282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824908f-a0cd-46da-bb2a-ab7dbfbcb0b2 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:22.698207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4e33a-2364-465b-9c82-204ead3c43c1 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:22.715592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88055822-2407-4ad1-86f4-a92ccba1f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:22.716456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88055822-2407-4ad1-86f4-a92ccba1f0f2 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:22.716874 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88055822-2407-4ad1-86f4-a92ccba1f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:22.717340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88055822-2407-4ad1-86f4-a92ccba1f0f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:22.728851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88055822-2407-4ad1-86f4-a92ccba1f0f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:22.730437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88055822-2407-4ad1-86f4-a92ccba1f0f2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:22.733020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de240dc8-5266-401f-8f48-f87d9c25e123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:22.734185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de240dc8-5266-401f-8f48-f87d9c25e123 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:22.734467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de240dc8-5266-401f-8f48-f87d9c25e123 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:22.734737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de240dc8-5266-401f-8f48-f87d9c25e123 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:22.735394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de240dc8-5266-401f-8f48-f87d9c25e123 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:22.736814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de240dc8-5266-401f-8f48-f87d9c25e123 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:22.739890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-88055822-2407-4ad1-86f4-a92ccba1f0f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:22.749448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:22.750458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:22.751355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:22.751355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:22.751355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:22.751355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 458/925] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:22.751947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dde3a-e0e0-4d0a-83c4-78056e387076 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:22.752058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dde3a-e0e0-4d0a-83c4-78056e387076 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:22.752306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dde3a-e0e0-4d0a-83c4-78056e387076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:22.762543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 468/926] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:39:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:39:22.763610 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de240dc8-5266-401f-8f48-f87d9c25e123 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:22.778679 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-1c1dde3a-e0e0-4d0a-83c4-78056e387076 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:22.779952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-1c1dde3a-e0e0-4d0a-83c4-78056e387076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:22.855912 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:22.872012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 459/927] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:22.872890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dde3a-e0e0-4d0a-83c4-78056e387076 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:22.916737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de240dc8-5266-401f-8f48-f87d9c25e123 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:22.968435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dddcd-05a4-4107-955d-921601c7c06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:22.969403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dddcd-05a4-4107-955d-921601c7c06a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:22.969828 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dddcd-05a4-4107-955d-921601c7c06a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:22.970567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dddcd-05a4-4107-955d-921601c7c06a None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:22.971656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dddcd-05a4-4107-955d-921601c7c06a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:22.972120 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dddcd-05a4-4107-955d-921601c7c06a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:22.973772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dddcd-05a4-4107-955d-921601c7c06a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:22.989774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 469/928] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:22.995649 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dddcd-05a4-4107-955d-921601c7c06a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:23.016471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dde3a-e0e0-4d0a-83c4-78056e387076 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:23.042770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:23.055144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c9f58-a907-4b86-ae03-4dde867213ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:23.055898 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c9f58-a907-4b86-ae03-4dde867213ef None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:23.056247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c9f58-a907-4b86-ae03-4dde867213ef None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:23.056593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c9f58-a907-4b86-ae03-4dde867213ef None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:23.057331 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c9f58-a907-4b86-ae03-4dde867213ef None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:23.058861 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c9f58-a907-4b86-ae03-4dde867213ef None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-9b1c9f58-a907-4b86-ae03-4dde867213ef None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:23.069244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:23.070381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:23.071404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 460/929] 10.222.0.36 () {56 vars in 1132 bytes} [Thu Aug 18 14:39:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:23.077686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c9f58-a907-4b86-ae03-4dde867213ef None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:23.128191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dddcd-05a4-4107-955d-921601c7c06a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:23.163660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:23.170369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6157145-2acb-4f71-ab49-663a3d692ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:23.171047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6157145-2acb-4f71-ab49-663a3d692ab0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:23.171323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6157145-2acb-4f71-ab49-663a3d692ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:23.171586 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6157145-2acb-4f71-ab49-663a3d692ab0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:23.178684 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6157145-2acb-4f71-ab49-663a3d692ab0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:23.180141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6157145-2acb-4f71-ab49-663a3d692ab0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-e6157145-2acb-4f71-ab49-663a3d692ab0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:23.191637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:23.192617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:23.198616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:23.198616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:23.198616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:23.198616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:23.198616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:23.198616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:23.198616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:23.198616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:23.198616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 14:39:23.198616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:23.199159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c9f58-a907-4b86-ae03-4dde867213ef None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:23.199244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 470/930] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Aug 18 14:39:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:23.199244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6157145-2acb-4f71-ab49-663a3d692ab0 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:23.231187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6208be3-acd6-49c6-b783-aecec7b386e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:23.231845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6208be3-acd6-49c6-b783-aecec7b386e9 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:23.232124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6208be3-acd6-49c6-b783-aecec7b386e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:23.232391 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6208be3-acd6-49c6-b783-aecec7b386e9 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:23.233007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6208be3-acd6-49c6-b783-aecec7b386e9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:23.233300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6208be3-acd6-49c6-b783-aecec7b386e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:23.234686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6208be3-acd6-49c6-b783-aecec7b386e9 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:23.239255 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 461/931] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Aug 18 14:39:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:23.240065 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6208be3-acd6-49c6-b783-aecec7b386e9 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:23.293245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:23.314056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6157145-2acb-4f71-ab49-663a3d692ab0 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:23.343818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6208be3-acd6-49c6-b783-aecec7b386e9 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:23.353877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92548a10-140f-4ecc-afa2-1ea7169b4634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:23.354508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92548a10-140f-4ecc-afa2-1ea7169b4634 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:23.354789 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92548a10-140f-4ecc-afa2-1ea7169b4634 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:23.355063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92548a10-140f-4ecc-afa2-1ea7169b4634 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:23.355644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92548a10-140f-4ecc-afa2-1ea7169b4634 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:23.357078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92548a10-140f-4ecc-afa2-1ea7169b4634 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-92548a10-140f-4ecc-afa2-1ea7169b4634 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:23.365778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:23.366794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:23.367738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 471/932] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Aug 18 14:39:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:23.369042 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92548a10-140f-4ecc-afa2-1ea7169b4634 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:23.386717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc252eb-29e0-4a97-9f1b-3046ddd410fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:23.387642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc252eb-29e0-4a97-9f1b-3046ddd410fc None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:23.388111 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc252eb-29e0-4a97-9f1b-3046ddd410fc None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:23.388525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc252eb-29e0-4a97-9f1b-3046ddd410fc None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:23.389396 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc252eb-29e0-4a97-9f1b-3046ddd410fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:23.390524 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc252eb-29e0-4a97-9f1b-3046ddd410fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:23.392322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc252eb-29e0-4a97-9f1b-3046ddd410fc None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:23.417243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 462/933] 10.222.0.36 () {56 vars in 1149 bytes} [Thu Aug 18 14:39:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:23.417243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc252eb-29e0-4a97-9f1b-3046ddd410fc None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:23.430208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:23.477124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92548a10-140f-4ecc-afa2-1ea7169b4634 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:23.511784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48e8f2-ef69-49ee-ab5e-a969952468a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:23.512305 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48e8f2-ef69-49ee-ab5e-a969952468a0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:23.512582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48e8f2-ef69-49ee-ab5e-a969952468a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:23.512851 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48e8f2-ef69-49ee-ab5e-a969952468a0 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:23.513472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48e8f2-ef69-49ee-ab5e-a969952468a0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:23.513770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48e8f2-ef69-49ee-ab5e-a969952468a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:23.515730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48e8f2-ef69-49ee-ab5e-a969952468a0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:23.520498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 472/934] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Aug 18 14:39:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:23.524356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc252eb-29e0-4a97-9f1b-3046ddd410fc None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:23.524747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48e8f2-ef69-49ee-ab5e-a969952468a0 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:23.535588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:23.560390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412750b-bb35-42e2-ab47-85dc5828a5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:23.560983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412750b-bb35-42e2-ab47-85dc5828a5ac None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:23.561264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412750b-bb35-42e2-ab47-85dc5828a5ac None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:23.562055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412750b-bb35-42e2-ab47-85dc5828a5ac None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:23.562676 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412750b-bb35-42e2-ab47-85dc5828a5ac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:23.562976 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412750b-bb35-42e2-ab47-85dc5828a5ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:23.564366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412750b-bb35-42e2-ab47-85dc5828a5ac None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:23.569231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 463/935] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Aug 18 14:39:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:23.574669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412750b-bb35-42e2-ab47-85dc5828a5ac None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:23.632619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48e8f2-ef69-49ee-ab5e-a969952468a0 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:23.650796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:23.668698 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3c12d-8122-4a28-852a-df892436e44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:23.669322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3c12d-8122-4a28-852a-df892436e44b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:23.669603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3c12d-8122-4a28-852a-df892436e44b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:23.669872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3c12d-8122-4a28-852a-df892436e44b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:23.676980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412750b-bb35-42e2-ab47-85dc5828a5ac None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:23.680726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3c12d-8122-4a28-852a-df892436e44b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:23.682161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3c12d-8122-4a28-852a-df892436e44b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-c2d3c12d-8122-4a28-852a-df892436e44b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:23.697086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:23.698072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:23.700031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:23.700031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:23.700031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:23.700031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 473/936] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:23.700031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3c12d-8122-4a28-852a-df892436e44b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:23.718839 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40680f2-e7c5-431a-abf9-61ad00bd2e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:23.719453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40680f2-e7c5-431a-abf9-61ad00bd2e58 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:23.719761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40680f2-e7c5-431a-abf9-61ad00bd2e58 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:23.720036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40680f2-e7c5-431a-abf9-61ad00bd2e58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:23.720697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40680f2-e7c5-431a-abf9-61ad00bd2e58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:23.722066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40680f2-e7c5-431a-abf9-61ad00bd2e58 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:23.752787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 464/937] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:39:23 2022] POST /identity/v3/endpoints => generated 347 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:23.752787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40680f2-e7c5-431a-abf9-61ad00bd2e58 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:23.784295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:23.833881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3c12d-8122-4a28-852a-df892436e44b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:23.883586 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8134ed2-9d45-4139-89bf-f0da53a6ab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:23.884405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8134ed2-9d45-4139-89bf-f0da53a6ab73 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:23.884786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8134ed2-9d45-4139-89bf-f0da53a6ab73 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:23.885324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8134ed2-9d45-4139-89bf-f0da53a6ab73 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:23.886168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8134ed2-9d45-4139-89bf-f0da53a6ab73 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:23.886576 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8134ed2-9d45-4139-89bf-f0da53a6ab73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:23.888212 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8134ed2-9d45-4139-89bf-f0da53a6ab73 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:23.893313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40680f2-e7c5-431a-abf9-61ad00bd2e58 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:23.905260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 474/938] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Aug 18 14:39:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:23.907245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8134ed2-9d45-4139-89bf-f0da53a6ab73 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:23.942719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8ab9f-d446-4f1d-9e00-34aa270431cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:23.943457 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8ab9f-d446-4f1d-9e00-34aa270431cc None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:23.944129 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8ab9f-d446-4f1d-9e00-34aa270431cc None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:23.944438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8ab9f-d446-4f1d-9e00-34aa270431cc None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:23.945113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8ab9f-d446-4f1d-9e00-34aa270431cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:23.945412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8ab9f-d446-4f1d-9e00-34aa270431cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:23.946837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8ab9f-d446-4f1d-9e00-34aa270431cc None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:23.950036 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:23.963295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 465/939] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:23.969166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8ab9f-d446-4f1d-9e00-34aa270431cc None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:24.042076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8134ed2-9d45-4139-89bf-f0da53a6ab73 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:24.073968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:24.083357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8ab9f-d446-4f1d-9e00-34aa270431cc None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:24.089652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f6274-17f2-4355-b02b-0dcdc8b1d821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:24.090383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f6274-17f2-4355-b02b-0dcdc8b1d821 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:24.090765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f6274-17f2-4355-b02b-0dcdc8b1d821 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:24.091136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f6274-17f2-4355-b02b-0dcdc8b1d821 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:24.091967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f6274-17f2-4355-b02b-0dcdc8b1d821 None admin] RBAC: Adding query filter params (interface=public, service_id=888f25e295844ee6b5a292855947eec7, region_id=RegionOne) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:24.093417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f6274-17f2-4355-b02b-0dcdc8b1d821 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:24.095379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f6274-17f2-4355-b02b-0dcdc8b1d821 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:24.102944 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 475/940] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Aug 18 14:39:23 2022] GET /identity/v3/endpoints?service_id=888f25e295844ee6b5a292855947eec7&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:24.108139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f6274-17f2-4355-b02b-0dcdc8b1d821 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:24.125079 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b387f8-6e43-483e-ac7e-b1c8eae421f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:24.125851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b387f8-6e43-483e-ac7e-b1c8eae421f4 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:24.126228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b387f8-6e43-483e-ac7e-b1c8eae421f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:24.126601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b387f8-6e43-483e-ac7e-b1c8eae421f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:24.134041 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b387f8-6e43-483e-ac7e-b1c8eae421f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:24.135603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b387f8-6e43-483e-ac7e-b1c8eae421f4 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-46b387f8-6e43-483e-ac7e-b1c8eae421f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:24.147225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:24.148320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:24.149402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:24.149402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:24.149402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:24.149402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:24.149402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:24.149402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:24.149402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:24.149402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:24.151716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 466/941] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:24.157268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b387f8-6e43-483e-ac7e-b1c8eae421f4 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:24.221684 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:24.231343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f6274-17f2-4355-b02b-0dcdc8b1d821 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:24.281060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf029641-b62a-4978-b516-89dface7dee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:24.281731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf029641-b62a-4978-b516-89dface7dee3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:24.282058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf029641-b62a-4978-b516-89dface7dee3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:24.282333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf029641-b62a-4978-b516-89dface7dee3 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:24.282934 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf029641-b62a-4978-b516-89dface7dee3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:24.284327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf029641-b62a-4978-b516-89dface7dee3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:24.286573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b387f8-6e43-483e-ac7e-b1c8eae421f4 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:24.296822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 476/942] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Aug 18 14:39:24 2022] GET /identity/v3/services => generated 1497 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:39:24.324634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f5d5f-bf9b-41c4-b8b5-9f539ef39a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:24.325629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f5d5f-bf9b-41c4-b8b5-9f539ef39a32 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:24.326117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f5d5f-bf9b-41c4-b8b5-9f539ef39a32 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:24.326596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f5d5f-bf9b-41c4-b8b5-9f539ef39a32 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:24.327589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f5d5f-bf9b-41c4-b8b5-9f539ef39a32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:24.328137 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f5d5f-bf9b-41c4-b8b5-9f539ef39a32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:24.329829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f5d5f-bf9b-41c4-b8b5-9f539ef39a32 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:24.345231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 467/943] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:24.351088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf029641-b62a-4978-b516-89dface7dee3 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:24.463549 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf029641-b62a-4978-b516-89dface7dee3 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:24.500310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df01e7-3119-40d1-972f-1f88c42349e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:24.501129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df01e7-3119-40d1-972f-1f88c42349e6 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:24.501512 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df01e7-3119-40d1-972f-1f88c42349e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:24.501890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df01e7-3119-40d1-972f-1f88c42349e6 None admin] PATH_INFO: `/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/cbb13837e1694d7aa928ac855e21d1a9/roles/b5a4a4e7850147889e8dfd1344c75440` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:24.504174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df01e7-3119-40d1-972f-1f88c42349e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=12f9875f45a84e92a848b8fc1636d6c6, user_id=cbb13837e1694d7aa928ac855e21d1a9, role_id=b5a4a4e7850147889e8dfd1344c75440)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:24.505722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df01e7-3119-40d1-972f-1f88c42349e6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:24.521509 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 477/944] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:39:24 2022] PUT /identity/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/cbb13837e1694d7aa928ac855e21d1a9/roles/b5a4a4e7850147889e8dfd1344c75440 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:39:24.817665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94350e1f-9a3b-491a-be03-fa310b95c90c None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:24.818221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94350e1f-9a3b-491a-be03-fa310b95c90c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:24.818221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94350e1f-9a3b-491a-be03-fa310b95c90c None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:24.819844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 468/945] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:24.825207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836971c5-9832-4d5e-9ee6-ed69cdcc9c10 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:24.825629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836971c5-9832-4d5e-9ee6-ed69cdcc9c10 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:24.825810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836971c5-9832-4d5e-9ee6-ed69cdcc9c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:24.844772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-836971c5-9832-4d5e-9ee6-ed69cdcc9c10 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:24.845971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-836971c5-9832-4d5e-9ee6-ed69cdcc9c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:24.860736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:24.922654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 478/946] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:25.192118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94350e1f-9a3b-491a-be03-fa310b95c90c None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:25.205289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:25.301247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94350e1f-9a3b-491a-be03-fa310b95c90c None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:25.335385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea637a-45de-4b08-8cc9-519dcb425cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:25.336010 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea637a-45de-4b08-8cc9-519dcb425cab None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:25.336285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea637a-45de-4b08-8cc9-519dcb425cab None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:25.336551 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea637a-45de-4b08-8cc9-519dcb425cab None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:25.337188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea637a-45de-4b08-8cc9-519dcb425cab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:25.338607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea637a-45de-4b08-8cc9-519dcb425cab None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:25.341199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: WARNING keystone.server.flask.common [None req-95ea637a-45de-4b08-8cc9-519dcb425cab None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 18 14:39:25.360846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.sql.core [None req-95ea637a-45de-4b08-8cc9-519dcb425cab None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 14:39:25.360846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 18 14:39:25.360846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [parameters: {'id': 'b9cf361adb61481a9bf41191c8d60911', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 18 14:39:25.360846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93178) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-95ea637a-45de-4b08-8cc9-519dcb425cab None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b9cf361adb61481a9bf41191c8d60911.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b9cf361adb61481a9bf41191c8d60911. Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application conn.query(q) Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 14:39:25.378816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application result.read() Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 18 14:39:25.379702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application next(self.gen) Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application yield resource Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application next(self.gen) Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise self.value Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application session.commit() Aug 18 14:39:25.380654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self.session.flush() Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application compat.raise_( Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 14:39:25.381556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 14:39:25.382489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application conn.query(q) Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application result.read() Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 14:39:25.383309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [parameters: {'id': 'b9cf361adb61481a9bf41191c8d60911', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:25.384233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:25.385018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:25.385950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:25.385950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 18 14:39:25.385950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 18 14:39:25.385950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:25.385950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:25.385950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 18 14:39:25.385950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 18 14:39:25.385950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b9cf361adb61481a9bf41191c8d60911. Aug 18 14:39:25.385950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:25.385950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 469/947] 10.222.0.36 () {60 vars in 1140 bytes} [Thu Aug 18 14:39:25 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:25.386487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836971c5-9832-4d5e-9ee6-ed69cdcc9c10 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:25.496270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836971c5-9832-4d5e-9ee6-ed69cdcc9c10 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:25.534124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7c1d2-d748-4ef0-a994-819d884e7288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:25.534762 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7c1d2-d748-4ef0-a994-819d884e7288 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:25.535043 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7c1d2-d748-4ef0-a994-819d884e7288 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:25.535320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7c1d2-d748-4ef0-a994-819d884e7288 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:25.542253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7c1d2-d748-4ef0-a994-819d884e7288 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:25.543713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7c1d2-d748-4ef0-a994-819d884e7288 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-7dc7c1d2-d748-4ef0-a994-819d884e7288 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:25.555499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:25.557634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:25.559805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:25.559805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:25.559805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:25.559805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:25.559805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:25.559805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:25.559805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:25.559805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:25.559805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 479/948] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:25.563629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea637a-45de-4b08-8cc9-519dcb425cab None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:25.634969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:25.673756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea637a-45de-4b08-8cc9-519dcb425cab None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:25.708298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762de9d8-a9f5-4f83-872b-f581495c6278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:25.708911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762de9d8-a9f5-4f83-872b-f581495c6278 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:25.709191 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762de9d8-a9f5-4f83-872b-f581495c6278 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:25.709464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762de9d8-a9f5-4f83-872b-f581495c6278 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:25.710124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762de9d8-a9f5-4f83-872b-f581495c6278 None admin] RBAC: Adding query filter params (name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:25.710489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762de9d8-a9f5-4f83-872b-f581495c6278 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:25.712206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762de9d8-a9f5-4f83-872b-f581495c6278 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:25.734077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 470/949] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:39:25 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:25.812347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7120365-9d7b-49cd-9ada-14bdcc167a3c None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:25.812651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7120365-9d7b-49cd-9ada-14bdcc167a3c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:25.812813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7120365-9d7b-49cd-9ada-14bdcc167a3c None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:25.814436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 480/950] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:25.820414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70c30b-8b81-466a-a012-1f7f06828e28 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:25.821035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70c30b-8b81-466a-a012-1f7f06828e28 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:25.821035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70c30b-8b81-466a-a012-1f7f06828e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:25.842415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-ae70c30b-8b81-466a-a012-1f7f06828e28 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:25.844902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ae70c30b-8b81-466a-a012-1f7f06828e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:25.929350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 471/951] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:26.203765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7120365-9d7b-49cd-9ada-14bdcc167a3c None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:26.216125 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:26.314149 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7120365-9d7b-49cd-9ada-14bdcc167a3c None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:26.351453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b46ca-5755-4016-9087-e02ff23f55dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:26.352128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b46ca-5755-4016-9087-e02ff23f55dc None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:26.352411 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b46ca-5755-4016-9087-e02ff23f55dc None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:26.352689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b46ca-5755-4016-9087-e02ff23f55dc None admin] PATH_INFO: `/v3/services/compute` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:26.353277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b46ca-5755-4016-9087-e02ff23f55dc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:26.354747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b46ca-5755-4016-9087-e02ff23f55dc None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-c59b46ca-5755-4016-9087-e02ff23f55dc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:26.364034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:26.365025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:26.366077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 481/952] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:26.370534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457371f9-f747-4d6f-99dc-782f14132a79 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:26.370820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457371f9-f747-4d6f-99dc-782f14132a79 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:26.371169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457371f9-f747-4d6f-99dc-782f14132a79 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:26.372781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 472/953] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:26.374583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b46ca-5755-4016-9087-e02ff23f55dc None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:26.380628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb5fc4-9bbb-4ca7-bbd7-d1b20bd4b470 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:26.381165 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb5fc4-9bbb-4ca7-bbd7-d1b20bd4b470 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:26.381663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb5fc4-9bbb-4ca7-bbd7-d1b20bd4b470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:26.405779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-f7eb5fc4-9bbb-4ca7-bbd7-d1b20bd4b470 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:26.407893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f7eb5fc4-9bbb-4ca7-bbd7-d1b20bd4b470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:26.454729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:26.479754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:26.487882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b46ca-5755-4016-9087-e02ff23f55dc None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:26.498402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 473/954] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:26.500065 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7ee81-ced5-4f15-bcaf-2b9d882194ee None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:26.500349 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7ee81-ced5-4f15-bcaf-2b9d882194ee None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:26.500620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7ee81-ced5-4f15-bcaf-2b9d882194ee None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:26.501989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 474/955] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:26.506959 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b361d-ed63-4808-8cad-0d9beda8d84e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:26.508566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b361d-ed63-4808-8cad-0d9beda8d84e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:26.508566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b361d-ed63-4808-8cad-0d9beda8d84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:26.529695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-d07b361d-ed63-4808-8cad-0d9beda8d84e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:26.530994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d07b361d-ed63-4808-8cad-0d9beda8d84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:26.533400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69cbc9-94a0-49a3-b24c-f3ff31786df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:26.535121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69cbc9-94a0-49a3-b24c-f3ff31786df7 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:26.535605 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69cbc9-94a0-49a3-b24c-f3ff31786df7 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:26.536112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69cbc9-94a0-49a3-b24c-f3ff31786df7 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:26.536949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69cbc9-94a0-49a3-b24c-f3ff31786df7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:26.537460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69cbc9-94a0-49a3-b24c-f3ff31786df7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:26.539526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69cbc9-94a0-49a3-b24c-f3ff31786df7 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:26.546678 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 482/956] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:39:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:26.556568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69cbc9-94a0-49a3-b24c-f3ff31786df7 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:26.627833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 475/957] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:26.674994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69cbc9-94a0-49a3-b24c-f3ff31786df7 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:26.710691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807237ae-e806-4b5b-b672-907e2a6775b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:26.711658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807237ae-e806-4b5b-b672-907e2a6775b6 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:26.712192 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807237ae-e806-4b5b-b672-907e2a6775b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:26.712720 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807237ae-e806-4b5b-b672-907e2a6775b6 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:26.713564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807237ae-e806-4b5b-b672-907e2a6775b6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:26.714069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807237ae-e806-4b5b-b672-907e2a6775b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:26.715645 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807237ae-e806-4b5b-b672-907e2a6775b6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:26.718004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:26.727254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 483/958] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:39:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:26.773553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b361d-ed63-4808-8cad-0d9beda8d84e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:26.887523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b361d-ed63-4808-8cad-0d9beda8d84e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:26.914727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:26.922778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807237ae-e806-4b5b-b672-907e2a6775b6 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:26.934872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7291ebb-34c1-46b2-8278-8030e961836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:26.935931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7291ebb-34c1-46b2-8278-8030e961836e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:26.936389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7291ebb-34c1-46b2-8278-8030e961836e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:26.937521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7291ebb-34c1-46b2-8278-8030e961836e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:26.938382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7291ebb-34c1-46b2-8278-8030e961836e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:26.940023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7291ebb-34c1-46b2-8278-8030e961836e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-b7291ebb-34c1-46b2-8278-8030e961836e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:26.952869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:26.954027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:26.954916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:26.957083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 476/959] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Aug 18 14:39:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:26.963239 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7291ebb-34c1-46b2-8278-8030e961836e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:27.054089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807237ae-e806-4b5b-b672-907e2a6775b6 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:27.095413 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7291ebb-34c1-46b2-8278-8030e961836e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:27.096460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe9180-fe77-44e8-b7d2-e94981af8f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:27.097293 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe9180-fe77-44e8-b7d2-e94981af8f40 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:27.097570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe9180-fe77-44e8-b7d2-e94981af8f40 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:27.098626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe9180-fe77-44e8-b7d2-e94981af8f40 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:27.111340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe9180-fe77-44e8-b7d2-e94981af8f40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-59fe9180-fe77-44e8-b7d2-e94981af8f40 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:27.118109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:27.119035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:27.119896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:27.119896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:27.119896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:27.119896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:27.119896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:27.119896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:27.119896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:27.119896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:27.119896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:27.119896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:27.119896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 14:39:27.119896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:27.120564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 484/960] 10.222.0.36 () {56 vars in 1120 bytes} [Thu Aug 18 14:39:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:27.126686 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe9180-fe77-44e8-b7d2-e94981af8f40 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:27.140495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1cf99-c7ae-4ad6-8759-4b690cc8b5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:27.141447 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1cf99-c7ae-4ad6-8759-4b690cc8b5cd None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:27.141922 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1cf99-c7ae-4ad6-8759-4b690cc8b5cd None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:27.142409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1cf99-c7ae-4ad6-8759-4b690cc8b5cd None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:27.143263 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1cf99-c7ae-4ad6-8759-4b690cc8b5cd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:27.143782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1cf99-c7ae-4ad6-8759-4b690cc8b5cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:27.145381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1cf99-c7ae-4ad6-8759-4b690cc8b5cd None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:27.156034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 477/961] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Aug 18 14:39:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:27.157110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1cf99-c7ae-4ad6-8759-4b690cc8b5cd None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:27.192813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:27.206456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:27.238932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe9180-fe77-44e8-b7d2-e94981af8f40 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:27.266541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1cf99-c7ae-4ad6-8759-4b690cc8b5cd None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:27.274628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024817a-f936-4632-836f-351675f1c555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:27.275329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024817a-f936-4632-836f-351675f1c555 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:27.275525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024817a-f936-4632-836f-351675f1c555 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:27.275840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024817a-f936-4632-836f-351675f1c555 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:27.276514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024817a-f936-4632-836f-351675f1c555 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:27.276820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024817a-f936-4632-836f-351675f1c555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:27.278382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024817a-f936-4632-836f-351675f1c555 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:27.292025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 485/962] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Aug 18 14:39:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:27.292918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024817a-f936-4632-836f-351675f1c555 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:27.305957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a499d5-0679-4292-9cb0-e5f17f204d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:27.306871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a499d5-0679-4292-9cb0-e5f17f204d5b None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:27.307348 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a499d5-0679-4292-9cb0-e5f17f204d5b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:27.307896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a499d5-0679-4292-9cb0-e5f17f204d5b None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:27.308731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a499d5-0679-4292-9cb0-e5f17f204d5b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:27.309242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a499d5-0679-4292-9cb0-e5f17f204d5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:27.310792 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a499d5-0679-4292-9cb0-e5f17f204d5b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:27.321119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 478/963] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Aug 18 14:39:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:27.322298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a499d5-0679-4292-9cb0-e5f17f204d5b None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:27.402129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024817a-f936-4632-836f-351675f1c555 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:27.427392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:27.430915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a499d5-0679-4292-9cb0-e5f17f204d5b None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:27.441331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e87474-28f6-4f70-9b60-cc6b2b3bc4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:27.441756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:27.441971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e87474-28f6-4f70-9b60-cc6b2b3bc4ea None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:27.442252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e87474-28f6-4f70-9b60-cc6b2b3bc4ea None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:27.442526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e87474-28f6-4f70-9b60-cc6b2b3bc4ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:27.456962 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e87474-28f6-4f70-9b60-cc6b2b3bc4ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:27.458493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e87474-28f6-4f70-9b60-cc6b2b3bc4ea None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:27.470937 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39914fbc-d9ce-4e89-9b72-aa807e984192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:27.471846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39914fbc-d9ce-4e89-9b72-aa807e984192 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:27.472326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39914fbc-d9ce-4e89-9b72-aa807e984192 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:27.472887 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39914fbc-d9ce-4e89-9b72-aa807e984192 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:27.473758 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39914fbc-d9ce-4e89-9b72-aa807e984192 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-59e87474-28f6-4f70-9b60-cc6b2b3bc4ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:27.474391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:27.475318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:27.476255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:27.476255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:27.476255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:27.477830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39914fbc-d9ce-4e89-9b72-aa807e984192 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:27.480053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 486/964] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:27.481544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e87474-28f6-4f70-9b60-cc6b2b3bc4ea None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:27.515860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 479/965] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:39:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:27.599870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e87474-28f6-4f70-9b60-cc6b2b3bc4ea None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:27.644885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd080529-c51f-4452-a07a-ea00f7a3ee71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:27.645508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd080529-c51f-4452-a07a-ea00f7a3ee71 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:27.645791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd080529-c51f-4452-a07a-ea00f7a3ee71 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:27.646066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd080529-c51f-4452-a07a-ea00f7a3ee71 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:27.646704 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd080529-c51f-4452-a07a-ea00f7a3ee71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:27.647012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd080529-c51f-4452-a07a-ea00f7a3ee71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:27.649690 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd080529-c51f-4452-a07a-ea00f7a3ee71 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:27.662312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 487/966] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:27.667926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39914fbc-d9ce-4e89-9b72-aa807e984192 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:27.773175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39914fbc-d9ce-4e89-9b72-aa807e984192 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:27.807893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabaa4ea-e978-413f-a87e-e19876f538f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:27.808526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaa4ea-e978-413f-a87e-e19876f538f2 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:27.808800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaa4ea-e978-413f-a87e-e19876f538f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:27.809066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaa4ea-e978-413f-a87e-e19876f538f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:27.812815 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3714a3f-e38c-4edf-905a-c4a8606a5929 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:27.813102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3714a3f-e38c-4edf-905a-c4a8606a5929 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:27.813375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3714a3f-e38c-4edf-905a-c4a8606a5929 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:27.815932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 488/967] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:39:27.817213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabaa4ea-e978-413f-a87e-e19876f538f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:27.818752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabaa4ea-e978-413f-a87e-e19876f538f2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:27.820224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694f857-4387-42ce-b330-cfc405bf45aa None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:27.820712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694f857-4387-42ce-b330-cfc405bf45aa None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:27.821202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694f857-4387-42ce-b330-cfc405bf45aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-dabaa4ea-e978-413f-a87e-e19876f538f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:27.830953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:27.832332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:27.833398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:27.833398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:27.833398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:27.833398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:27.833398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:27.833398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:27.833398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:27.833398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:27.833398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 14:39:27.833398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:27.833398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 480/968] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Aug 18 14:39:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:27.840221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabaa4ea-e978-413f-a87e-e19876f538f2 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:27.848479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-6694f857-4387-42ce-b330-cfc405bf45aa None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:27.849965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6694f857-4387-42ce-b330-cfc405bf45aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:27.868168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:27.882490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:27.951731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 489/969] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:27.970837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabaa4ea-e978-413f-a87e-e19876f538f2 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:28.006641 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee91c0c-91f0-439b-bb58-ccb827682ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:28.007468 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee91c0c-91f0-439b-bb58-ccb827682ad2 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:28.007956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee91c0c-91f0-439b-bb58-ccb827682ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:28.008399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee91c0c-91f0-439b-bb58-ccb827682ad2 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:28.009276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee91c0c-91f0-439b-bb58-ccb827682ad2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:28.009721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee91c0c-91f0-439b-bb58-ccb827682ad2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:28.011336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee91c0c-91f0-439b-bb58-ccb827682ad2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:28.030511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 481/970] 10.222.0.36 () {56 vars in 1149 bytes} [Thu Aug 18 14:39:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:28.036308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694f857-4387-42ce-b330-cfc405bf45aa None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:28.136627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694f857-4387-42ce-b330-cfc405bf45aa None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:28.174957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04b456-03b8-4e56-ad86-ef128777d8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:28.175558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04b456-03b8-4e56-ad86-ef128777d8f2 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:28.175864 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04b456-03b8-4e56-ad86-ef128777d8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:28.176136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04b456-03b8-4e56-ad86-ef128777d8f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:28.212059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:28.246933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee91c0c-91f0-439b-bb58-ccb827682ad2 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:28.247770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04b456-03b8-4e56-ad86-ef128777d8f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:28.249919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04b456-03b8-4e56-ad86-ef128777d8f2 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:28.333183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:28.353479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 490/971] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:28 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:39:28.355014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04b456-03b8-4e56-ad86-ef128777d8f2 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:28.359892 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee91c0c-91f0-439b-bb58-ccb827682ad2 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:28.396494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06efbd-e86a-414e-bc72-12af72f1b3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:28.397544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06efbd-e86a-414e-bc72-12af72f1b3d8 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:28.398041 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06efbd-e86a-414e-bc72-12af72f1b3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:28.398635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06efbd-e86a-414e-bc72-12af72f1b3d8 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:28.399456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06efbd-e86a-414e-bc72-12af72f1b3d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:28.401107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06efbd-e86a-414e-bc72-12af72f1b3d8 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:28.415410 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:28.418018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.sql.core [None req-5e06efbd-e86a-414e-bc72-12af72f1b3d8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 14:39:28.418018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 18 14:39:28.418018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [parameters: {'user_id': 'ebebb34585834f2ba3710113b53dfd23', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 18 14:39:28.418018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93178) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-5e06efbd-e86a-414e-bc72-12af72f1b3d8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application conn.query(q) Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application result.read() Aug 18 14:39:28.426672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 14:39:28.427544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application next(self.gen) Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application yield resource Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application next(self.gen) Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise self.value Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application session.commit() Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 14:39:28.428535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self.session.flush() Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application compat.raise_( Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 14:39:28.429427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 14:39:28.430343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application conn.query(q) Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application result.read() Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 14:39:28.431206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ebebb34585834f2ba3710113b53dfd23', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:28.433532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 18 14:39:28.435262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 18 14:39:28.438085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 18 14:39:28.438085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 18 14:39:28.438085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 14:39:28.438085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 14:39:28.438085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 18 14:39:28.438085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:28.445500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 482/972] 10.222.0.36 () {60 vars in 1131 bytes} [Thu Aug 18 14:39:28 2022] POST /identity/v3/users => generated 160 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:28.447269 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06efbd-e86a-414e-bc72-12af72f1b3d8 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:28.460453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04b456-03b8-4e56-ad86-ef128777d8f2 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:28.495161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3a6c2-109f-417a-a218-5eb9288439fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:28.495811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3a6c2-109f-417a-a218-5eb9288439fc None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:28.496086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3a6c2-109f-417a-a218-5eb9288439fc None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:28.496356 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3a6c2-109f-417a-a218-5eb9288439fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:28.507513 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3a6c2-109f-417a-a218-5eb9288439fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:28.509035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3a6c2-109f-417a-a218-5eb9288439fc None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-eef3a6c2-109f-417a-a218-5eb9288439fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:28.523391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:28.524414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:28.525319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:28.525319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:28.525319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:28.528992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 491/973] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:28.530533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3a6c2-109f-417a-a218-5eb9288439fc None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:28.564483 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06efbd-e86a-414e-bc72-12af72f1b3d8 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:28.603602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d86117-a969-47e2-b1ae-59e6b070178e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:28.604617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d86117-a969-47e2-b1ae-59e6b070178e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:28.605093 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d86117-a969-47e2-b1ae-59e6b070178e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:28.605560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d86117-a969-47e2-b1ae-59e6b070178e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:28.614655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d86117-a969-47e2-b1ae-59e6b070178e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:28.616753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d86117-a969-47e2-b1ae-59e6b070178e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-07d86117-a969-47e2-b1ae-59e6b070178e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:28.629845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:28.630828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:28.631783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:28.631783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:28.631783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:28.631783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:28.631783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:28.631783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:28.631783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:28.631783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:28.631783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 14:39:28.631783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:28.638862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3a6c2-109f-417a-a218-5eb9288439fc None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:28.639378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 483/974] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:39:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:28.640495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d86117-a969-47e2-b1ae-59e6b070178e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:28.644696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:28.649644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:28.674607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab1cef-4d0c-4b86-97d1-c861b0ce579d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:28.675214 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab1cef-4d0c-4b86-97d1-c861b0ce579d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:28.675489 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab1cef-4d0c-4b86-97d1-c861b0ce579d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:28.675786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab1cef-4d0c-4b86-97d1-c861b0ce579d None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:28.676434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab1cef-4d0c-4b86-97d1-c861b0ce579d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:28.676721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab1cef-4d0c-4b86-97d1-c861b0ce579d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:28.678129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab1cef-4d0c-4b86-97d1-c861b0ce579d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:28.695263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 492/975] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:28.696687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab1cef-4d0c-4b86-97d1-c861b0ce579d None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:28.756719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d86117-a969-47e2-b1ae-59e6b070178e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:28.790059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec1ac4-688c-455d-800d-df3462c92701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:28.790656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec1ac4-688c-455d-800d-df3462c92701 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:28.790932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec1ac4-688c-455d-800d-df3462c92701 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:28.791201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec1ac4-688c-455d-800d-df3462c92701 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:28.791898 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec1ac4-688c-455d-800d-df3462c92701 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:28.792197 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec1ac4-688c-455d-800d-df3462c92701 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:28.793609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec1ac4-688c-455d-800d-df3462c92701 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:28.800770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab1cef-4d0c-4b86-97d1-c861b0ce579d None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:28.830453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 484/976] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Aug 18 14:39:28 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:28.831756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea0b18-bce4-4c40-b453-af828d82d3fd None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:28.832038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea0b18-bce4-4c40-b453-af828d82d3fd None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:28.832305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea0b18-bce4-4c40-b453-af828d82d3fd None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:28.833722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 485/977] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:28.839109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be2fca-792b-4888-9612-3a10cac7f4a4 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:28.839388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be2fca-792b-4888-9612-3a10cac7f4a4 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:28.839664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be2fca-792b-4888-9612-3a10cac7f4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:28.846215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baeec8d-476b-4a2c-99bc-c39bec3b78d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:28.847590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baeec8d-476b-4a2c-99bc-c39bec3b78d1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:28.847981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baeec8d-476b-4a2c-99bc-c39bec3b78d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:28.848252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baeec8d-476b-4a2c-99bc-c39bec3b78d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:28.856259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baeec8d-476b-4a2c-99bc-c39bec3b78d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:28.857784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baeec8d-476b-4a2c-99bc-c39bec3b78d1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:28.861061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-d2be2fca-792b-4888-9612-3a10cac7f4a4 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:28.863346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d2be2fca-792b-4888-9612-3a10cac7f4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-9baeec8d-476b-4a2c-99bc-c39bec3b78d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:28.872088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:28.873166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:28.874097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:28.874097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:28.874097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:28.874097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:28.874097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:28.874097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:28.874097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:28.874097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:28.878943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 493/978] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:28.880515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baeec8d-476b-4a2c-99bc-c39bec3b78d1 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:28.928518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:28.947421 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:28.971014 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 486/979] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:29.009814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baeec8d-476b-4a2c-99bc-c39bec3b78d1 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:29.047356 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee3b16-fcc8-477c-8533-d249fa6a7418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:29.047994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee3b16-fcc8-477c-8533-d249fa6a7418 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:29.048271 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee3b16-fcc8-477c-8533-d249fa6a7418 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:29.048537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee3b16-fcc8-477c-8533-d249fa6a7418 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:29.049232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee3b16-fcc8-477c-8533-d249fa6a7418 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:29.049532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee3b16-fcc8-477c-8533-d249fa6a7418 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:29.051010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee3b16-fcc8-477c-8533-d249fa6a7418 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:29.065799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 494/980] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:29.072487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be2fca-792b-4888-9612-3a10cac7f4a4 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:29.180821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be2fca-792b-4888-9612-3a10cac7f4a4 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:29.215686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e83911-9d67-4777-97bf-e9f33a0b30dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:29.216322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e83911-9d67-4777-97bf-e9f33a0b30dc None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:29.216599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e83911-9d67-4777-97bf-e9f33a0b30dc None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:29.216869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e83911-9d67-4777-97bf-e9f33a0b30dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:29.217584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e83911-9d67-4777-97bf-e9f33a0b30dc None admin] RBAC: Adding query filter params (role.id=b5a4a4e7850147889e8dfd1344c75440, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=cbb13837e1694d7aa928ac855e21d1a9) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:29.218456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e83911-9d67-4777-97bf-e9f33a0b30dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:29.219910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e83911-9d67-4777-97bf-e9f33a0b30dc None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:29.225924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 487/981] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:39:29 2022] GET /identity/v3/role_assignments?user.id=cbb13837e1694d7aa928ac855e21d1a9&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=b5a4a4e7850147889e8dfd1344c75440 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:29.241157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee3b16-fcc8-477c-8533-d249fa6a7418 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:29.342437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:29.354404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee3b16-fcc8-477c-8533-d249fa6a7418 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:29.395186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1508a0-fd6d-4f28-817c-79e4a49eaee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:29.395861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1508a0-fd6d-4f28-817c-79e4a49eaee2 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:29.396147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1508a0-fd6d-4f28-817c-79e4a49eaee2 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:29.396423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1508a0-fd6d-4f28-817c-79e4a49eaee2 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:29.397082 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1508a0-fd6d-4f28-817c-79e4a49eaee2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:29.398480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1508a0-fd6d-4f28-817c-79e4a49eaee2 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:29.423739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 495/982] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:39:29 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:29.506593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad3106-adfe-4396-be81-2faa4321bd03 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:29.506770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad3106-adfe-4396-be81-2faa4321bd03 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:29.507051 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad3106-adfe-4396-be81-2faa4321bd03 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:29.508632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 488/983] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:29.514123 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a9fe5-637b-4695-a8f0-5eee7ba6427e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:29.514418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a9fe5-637b-4695-a8f0-5eee7ba6427e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:29.515242 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a9fe5-637b-4695-a8f0-5eee7ba6427e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:29.534024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-e44a9fe5-637b-4695-a8f0-5eee7ba6427e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:29.535256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e44a9fe5-637b-4695-a8f0-5eee7ba6427e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:29.614389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 496/984] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:29.884622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad3106-adfe-4396-be81-2faa4321bd03 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:29.952704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:29.998971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad3106-adfe-4396-be81-2faa4321bd03 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:30.034212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359499ba-c84b-46fd-8a5c-cde74886c9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:30.034837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359499ba-c84b-46fd-8a5c-cde74886c9ed None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:30.035118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359499ba-c84b-46fd-8a5c-cde74886c9ed None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:30.035391 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359499ba-c84b-46fd-8a5c-cde74886c9ed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:30.036002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359499ba-c84b-46fd-8a5c-cde74886c9ed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:30.037368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359499ba-c84b-46fd-8a5c-cde74886c9ed None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-359499ba-c84b-46fd-8a5c-cde74886c9ed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:30.046164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:30.047203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:30.048526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 489/985] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Aug 18 14:39:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:30.053079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a9fe5-637b-4695-a8f0-5eee7ba6427e None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:30.131220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:30.162061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a9fe5-637b-4695-a8f0-5eee7ba6427e None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:30.196445 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392a779-1b36-4112-b497-37026b34874c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:30.197076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392a779-1b36-4112-b497-37026b34874c None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:30.197378 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392a779-1b36-4112-b497-37026b34874c None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:30.197711 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392a779-1b36-4112-b497-37026b34874c None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:30.198377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392a779-1b36-4112-b497-37026b34874c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:30.198795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392a779-1b36-4112-b497-37026b34874c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:30.200216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392a779-1b36-4112-b497-37026b34874c None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:30.205733 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 497/986] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Aug 18 14:39:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:30.211423 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359499ba-c84b-46fd-8a5c-cde74886c9ed None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:30.317282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359499ba-c84b-46fd-8a5c-cde74886c9ed None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:30.338512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:30.355843 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e03f70-81a8-4703-a0e0-e0a208694945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:30.356722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e03f70-81a8-4703-a0e0-e0a208694945 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:30.357007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e03f70-81a8-4703-a0e0-e0a208694945 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:30.357280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e03f70-81a8-4703-a0e0-e0a208694945 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:30.357946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e03f70-81a8-4703-a0e0-e0a208694945 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:30.358251 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e03f70-81a8-4703-a0e0-e0a208694945 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:30.359793 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e03f70-81a8-4703-a0e0-e0a208694945 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:30.370526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 490/987] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Aug 18 14:39:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:30.371790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392a779-1b36-4112-b497-37026b34874c None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:30.473062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392a779-1b36-4112-b497-37026b34874c None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:30.507613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fe988-2876-4c01-bda2-d70ca3602e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:30.508247 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fe988-2876-4c01-bda2-d70ca3602e99 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:30.508525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fe988-2876-4c01-bda2-d70ca3602e99 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:30.508796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fe988-2876-4c01-bda2-d70ca3602e99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:30.509510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fe988-2876-4c01-bda2-d70ca3602e99 None admin] RBAC: Adding query filter params (interface=admin, service_id=888f25e295844ee6b5a292855947eec7, region_id=RegionOne) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:30.509821 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fe988-2876-4c01-bda2-d70ca3602e99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:30.511181 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fe988-2876-4c01-bda2-d70ca3602e99 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:30.516323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 498/988] 10.222.0.36 () {56 vars in 1249 bytes} [Thu Aug 18 14:39:30 2022] GET /identity/v3/endpoints?service_id=888f25e295844ee6b5a292855947eec7&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:30.522019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e03f70-81a8-4703-a0e0-e0a208694945 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:30.620485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e03f70-81a8-4703-a0e0-e0a208694945 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:30.653083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea290e-5883-44ff-bd96-fe9770676abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:30.653696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea290e-5883-44ff-bd96-fe9770676abc None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:30.653954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea290e-5883-44ff-bd96-fe9770676abc None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:30.654221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea290e-5883-44ff-bd96-fe9770676abc None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:30.654815 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea290e-5883-44ff-bd96-fe9770676abc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:30.656223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea290e-5883-44ff-bd96-fe9770676abc None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:30.662430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 491/989] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Aug 18 14:39:30 2022] GET /identity/v3/services => generated 1727 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:39:31.207468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b910e-3f8b-4c42-86b0-09f5f58bcac0 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:31.207879 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b910e-3f8b-4c42-86b0-09f5f58bcac0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:31.208065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b910e-3f8b-4c42-86b0-09f5f58bcac0 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:31.209509 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 499/990] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:31.214475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59e3be-a98f-4da5-baa3-6c08f9503f54 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:31.214915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59e3be-a98f-4da5-baa3-6c08f9503f54 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:31.215000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59e3be-a98f-4da5-baa3-6c08f9503f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:31.233092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-6b59e3be-a98f-4da5-baa3-6c08f9503f54 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:31.234288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6b59e3be-a98f-4da5-baa3-6c08f9503f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:31.279535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:31.305149 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 492/991] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:31.344484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21467bc8-cc69-4f6f-aae7-c45a1768bc5e None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:31.344781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21467bc8-cc69-4f6f-aae7-c45a1768bc5e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:31.345044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21467bc8-cc69-4f6f-aae7-c45a1768bc5e None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:31.346548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 500/992] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:31.352554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4572f0-4b46-4e51-a103-2f75d200f101 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:31.352926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4572f0-4b46-4e51-a103-2f75d200f101 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:31.353201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4572f0-4b46-4e51-a103-2f75d200f101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:31.371598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-5c4572f0-4b46-4e51-a103-2f75d200f101 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:31.372824 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5c4572f0-4b46-4e51-a103-2f75d200f101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:31.439367 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 493/993] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:31.574477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21467bc8-cc69-4f6f-aae7-c45a1768bc5e None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:31.584619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:31.678520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21467bc8-cc69-4f6f-aae7-c45a1768bc5e None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:31.713243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cb048-748b-4025-bc07-e46abf3c0e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:31.713875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cb048-748b-4025-bc07-e46abf3c0e77 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:31.714147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cb048-748b-4025-bc07-e46abf3c0e77 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:31.714414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cb048-748b-4025-bc07-e46abf3c0e77 None admin] PATH_INFO: `/v3/services/image` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:31.714988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cb048-748b-4025-bc07-e46abf3c0e77 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:31.716461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cb048-748b-4025-bc07-e46abf3c0e77 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:31.716837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4572f0-4b46-4e51-a103-2f75d200f101 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-5f5cb048-748b-4025-bc07-e46abf3c0e77 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:31.725739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:31.727104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 14:39:31.728246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:31.735532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 501/994] 10.222.0.36 () {56 vars in 1105 bytes} [Thu Aug 18 14:39:31 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:31.737115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cb048-748b-4025-bc07-e46abf3c0e77 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:31.809197 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:31.821420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4572f0-4b46-4e51-a103-2f75d200f101 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:31.830781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:31.844537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cb048-748b-4025-bc07-e46abf3c0e77 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:31.857350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308f635-f24f-4c43-b4f0-5a3d4d3fc26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:31.858089 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308f635-f24f-4c43-b4f0-5a3d4d3fc26b None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:31.858977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308f635-f24f-4c43-b4f0-5a3d4d3fc26b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:31.859225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308f635-f24f-4c43-b4f0-5a3d4d3fc26b None admin] PATH_INFO: `/v3/services/compute` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:31.859839 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308f635-f24f-4c43-b4f0-5a3d4d3fc26b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:31.861320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308f635-f24f-4c43-b4f0-5a3d4d3fc26b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-5308f635-f24f-4c43-b4f0-5a3d4d3fc26b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:31.870748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:31.872132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:31.872997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 494/995] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:31.878064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308f635-f24f-4c43-b4f0-5a3d4d3fc26b None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:31.881291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a494a-d075-42ef-9a07-b33c9314d8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:31.881955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a494a-d075-42ef-9a07-b33c9314d8cf None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:31.882238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a494a-d075-42ef-9a07-b33c9314d8cf None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:31.882510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a494a-d075-42ef-9a07-b33c9314d8cf None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:31.883164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a494a-d075-42ef-9a07-b33c9314d8cf None admin] RBAC: Adding query filter params (name=image) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:31.883465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a494a-d075-42ef-9a07-b33c9314d8cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:31.884856 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a494a-d075-42ef-9a07-b33c9314d8cf None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:31.890064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 502/996] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:31.895636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a494a-d075-42ef-9a07-b33c9314d8cf None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:31.988133 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308f635-f24f-4c43-b4f0-5a3d4d3fc26b None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:32.002946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a494a-d075-42ef-9a07-b33c9314d8cf None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:32.023254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027d351-0646-4d8a-915f-e48ebd5a1c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:32.023889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027d351-0646-4d8a-915f-e48ebd5a1c40 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:32.024165 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027d351-0646-4d8a-915f-e48ebd5a1c40 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:32.024432 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027d351-0646-4d8a-915f-e48ebd5a1c40 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:32.025053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027d351-0646-4d8a-915f-e48ebd5a1c40 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:32.025350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027d351-0646-4d8a-915f-e48ebd5a1c40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:32.026706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027d351-0646-4d8a-915f-e48ebd5a1c40 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:32.032244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 495/997] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:39:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:32.038046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027d351-0646-4d8a-915f-e48ebd5a1c40 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:32.040144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04defaf-0383-43b7-a787-6f6838bc81c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:32.041042 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04defaf-0383-43b7-a787-6f6838bc81c8 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:32.041540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04defaf-0383-43b7-a787-6f6838bc81c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:32.041816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04defaf-0383-43b7-a787-6f6838bc81c8 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:32.042448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04defaf-0383-43b7-a787-6f6838bc81c8 None admin] RBAC: Adding query filter params (type=image) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:32.042752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04defaf-0383-43b7-a787-6f6838bc81c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:32.044200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04defaf-0383-43b7-a787-6f6838bc81c8 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:32.045488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:32.055914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 503/998] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:31 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:32.076309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:32.149930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027d351-0646-4d8a-915f-e48ebd5a1c40 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:32.185460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd59fb-85b5-47ac-a141-62f2179c6c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:32.186071 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd59fb-85b5-47ac-a141-62f2179c6c26 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:32.186344 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd59fb-85b5-47ac-a141-62f2179c6c26 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:32.186646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd59fb-85b5-47ac-a141-62f2179c6c26 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:32.187274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd59fb-85b5-47ac-a141-62f2179c6c26 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:32.187571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd59fb-85b5-47ac-a141-62f2179c6c26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:32.188971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd59fb-85b5-47ac-a141-62f2179c6c26 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:32.194451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 496/999] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:39:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:32.200461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04defaf-0383-43b7-a787-6f6838bc81c8 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:32.303673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04defaf-0383-43b7-a787-6f6838bc81c8 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:32.336551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a6171-9f19-4fcc-9def-5b547b6b054e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:32.337167 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a6171-9f19-4fcc-9def-5b547b6b054e None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:32.337443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a6171-9f19-4fcc-9def-5b547b6b054e None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:32.337710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a6171-9f19-4fcc-9def-5b547b6b054e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:32.338407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a6171-9f19-4fcc-9def-5b547b6b054e None admin] RBAC: Adding query filter params (interface=public, service_id=2cffe2e7ffc645348e18da8df146929d, region_id=RegionOne) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:32.338712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a6171-9f19-4fcc-9def-5b547b6b054e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:32.340100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a6171-9f19-4fcc-9def-5b547b6b054e None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:32.350235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 504/1000] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Aug 18 14:39:32 2022] GET /identity/v3/endpoints?service_id=2cffe2e7ffc645348e18da8df146929d&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:32.351827 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd59fb-85b5-47ac-a141-62f2179c6c26 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:32.454428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:32.458147 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd59fb-85b5-47ac-a141-62f2179c6c26 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:32.492617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f561976-c5c4-4865-9278-cead6db38221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:32.493226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f561976-c5c4-4865-9278-cead6db38221 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:32.493505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f561976-c5c4-4865-9278-cead6db38221 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:32.493777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f561976-c5c4-4865-9278-cead6db38221 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:32.494380 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f561976-c5c4-4865-9278-cead6db38221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:32.495778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f561976-c5c4-4865-9278-cead6db38221 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:32.508068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 497/1001] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Aug 18 14:39:32 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:39:32.586989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d6cf5-ae39-49ac-9141-8108a92412ef None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:32.587542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d6cf5-ae39-49ac-9141-8108a92412ef None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:32.587542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d6cf5-ae39-49ac-9141-8108a92412ef None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:32.589076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 505/1002] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:32.594566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acae541-bd76-4fbc-a61a-2cddc6937672 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:32.594870 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acae541-bd76-4fbc-a61a-2cddc6937672 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:32.595123 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acae541-bd76-4fbc-a61a-2cddc6937672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:32.614345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-5acae541-bd76-4fbc-a61a-2cddc6937672 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:32.615586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5acae541-bd76-4fbc-a61a-2cddc6937672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:32.687724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 498/1003] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:32.951726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d6cf5-ae39-49ac-9141-8108a92412ef None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:33.015492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:33.055921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d6cf5-ae39-49ac-9141-8108a92412ef None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:33.089459 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec9939-a5f9-4e7b-a53e-bd80fcd68348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:33.090083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9939-a5f9-4e7b-a53e-bd80fcd68348 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:33.090358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9939-a5f9-4e7b-a53e-bd80fcd68348 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:33.090631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9939-a5f9-4e7b-a53e-bd80fcd68348 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:33.091205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec9939-a5f9-4e7b-a53e-bd80fcd68348 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:33.093106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec9939-a5f9-4e7b-a53e-bd80fcd68348 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-efec9939-a5f9-4e7b-a53e-bd80fcd68348 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:33.101672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:33.102691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:33.103562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 506/1004] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Aug 18 14:39:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:33.109246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acae541-bd76-4fbc-a61a-2cddc6937672 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:33.201897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:33.209905 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acae541-bd76-4fbc-a61a-2cddc6937672 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:33.243392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb47ee-3b63-4ee8-8ecf-b21625e49dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:33.244009 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb47ee-3b63-4ee8-8ecf-b21625e49dff None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:33.244285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb47ee-3b63-4ee8-8ecf-b21625e49dff None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:33.244553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb47ee-3b63-4ee8-8ecf-b21625e49dff None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:33.245195 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb47ee-3b63-4ee8-8ecf-b21625e49dff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:33.245495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb47ee-3b63-4ee8-8ecf-b21625e49dff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:33.246851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb47ee-3b63-4ee8-8ecf-b21625e49dff None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:33.255813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 499/1005] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Aug 18 14:39:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:33.257937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec9939-a5f9-4e7b-a53e-bd80fcd68348 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:33.360094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec9939-a5f9-4e7b-a53e-bd80fcd68348 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:33.384822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:33.394255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e1436-700d-4106-9aee-aebf023ad8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:33.394862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e1436-700d-4106-9aee-aebf023ad8e3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:33.395138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e1436-700d-4106-9aee-aebf023ad8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:33.395407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e1436-700d-4106-9aee-aebf023ad8e3 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:33.396066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e1436-700d-4106-9aee-aebf023ad8e3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:33.396369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e1436-700d-4106-9aee-aebf023ad8e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:33.397740 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e1436-700d-4106-9aee-aebf023ad8e3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:33.402839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 507/1006] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Aug 18 14:39:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:33.409241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb47ee-3b63-4ee8-8ecf-b21625e49dff None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:33.520801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb47ee-3b63-4ee8-8ecf-b21625e49dff None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:33.556393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70220f10-3507-4d53-b70c-294bfc14bfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:33.557001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70220f10-3507-4d53-b70c-294bfc14bfce None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:33.557284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70220f10-3507-4d53-b70c-294bfc14bfce None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:33.557556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70220f10-3507-4d53-b70c-294bfc14bfce None admin] PATH_INFO: `/v3/endpoints` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:33.558224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70220f10-3507-4d53-b70c-294bfc14bfce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:33.559840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70220f10-3507-4d53-b70c-294bfc14bfce None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:33.578663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 500/1007] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:39:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:34.019491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf0690-618f-4aa5-b397-b87dcf65e69f None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:34.020042 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf0690-618f-4aa5-b397-b87dcf65e69f None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:34.020119 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf0690-618f-4aa5-b397-b87dcf65e69f None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:34.021716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 508/1008] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:34.026806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29826cbc-b91f-437a-80dd-4b5a0da356a3 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:34.027095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29826cbc-b91f-437a-80dd-4b5a0da356a3 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:34.027366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29826cbc-b91f-437a-80dd-4b5a0da356a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:34.049189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-29826cbc-b91f-437a-80dd-4b5a0da356a3 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:34.050380 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-29826cbc-b91f-437a-80dd-4b5a0da356a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:34.067179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:34.141575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 501/1009] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:34.410664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf0690-618f-4aa5-b397-b87dcf65e69f None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:34.460145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e43b6-4030-47a3-ba49-7699c1237662 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:34.460663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e43b6-4030-47a3-ba49-7699c1237662 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:34.460663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e43b6-4030-47a3-ba49-7699c1237662 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:34.462145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 502/1010] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:34.466710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aed1b6-57d3-496f-bad2-4ecf54aec2ed None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:34.466987 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aed1b6-57d3-496f-bad2-4ecf54aec2ed None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:34.467257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aed1b6-57d3-496f-bad2-4ecf54aec2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:34.485933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-79aed1b6-57d3-496f-bad2-4ecf54aec2ed None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:34.487112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-79aed1b6-57d3-496f-bad2-4ecf54aec2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:34.521328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf0690-618f-4aa5-b397-b87dcf65e69f None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:34.561212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 503/1011] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:34.561764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb0349-dafb-40b1-9107-ed1abaa3be7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:34.562461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb0349-dafb-40b1-9107-ed1abaa3be7e None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:34.562768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb0349-dafb-40b1-9107-ed1abaa3be7e None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:34.563090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb0349-dafb-40b1-9107-ed1abaa3be7e None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:34.563803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb0349-dafb-40b1-9107-ed1abaa3be7e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:34.565164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb0349-dafb-40b1-9107-ed1abaa3be7e None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:34.585030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 509/1012] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:39:34 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:34.829184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aed1b6-57d3-496f-bad2-4ecf54aec2ed None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:34.908430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:34.939329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aed1b6-57d3-496f-bad2-4ecf54aec2ed None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:34.980151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44260d30-cec4-40f0-9552-06ce5818f675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:34.980772 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44260d30-cec4-40f0-9552-06ce5818f675 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:34.981042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44260d30-cec4-40f0-9552-06ce5818f675 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:34.981307 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44260d30-cec4-40f0-9552-06ce5818f675 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:34.981879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44260d30-cec4-40f0-9552-06ce5818f675 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:34.983254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44260d30-cec4-40f0-9552-06ce5818f675 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-44260d30-cec4-40f0-9552-06ce5818f675 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:34.991781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:34.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 14:39:34.993655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:34.995756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 504/1013] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:34.998743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb0349-dafb-40b1-9107-ed1abaa3be7e None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:35.049993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:35.102840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb0349-dafb-40b1-9107-ed1abaa3be7e None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:35.137013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f122f9e-0afa-4557-bd84-e33c689c7dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:35.137616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f122f9e-0afa-4557-bd84-e33c689c7dae None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:35.137891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f122f9e-0afa-4557-bd84-e33c689c7dae None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:35.138161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f122f9e-0afa-4557-bd84-e33c689c7dae None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:35.138786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f122f9e-0afa-4557-bd84-e33c689c7dae None admin] RBAC: Adding query filter params (name=compute) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:35.139083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f122f9e-0afa-4557-bd84-e33c689c7dae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:35.140994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f122f9e-0afa-4557-bd84-e33c689c7dae None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:35.146002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 510/1014] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:39:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:35.151871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44260d30-cec4-40f0-9552-06ce5818f675 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:35.252114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44260d30-cec4-40f0-9552-06ce5818f675 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:35.287694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a845d7-0710-4f94-9b88-3a817cb25794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:35.288321 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a845d7-0710-4f94-9b88-3a817cb25794 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:35.288595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a845d7-0710-4f94-9b88-3a817cb25794 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:35.289044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a845d7-0710-4f94-9b88-3a817cb25794 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:35.289692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a845d7-0710-4f94-9b88-3a817cb25794 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:35.289963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a845d7-0710-4f94-9b88-3a817cb25794 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:35.291312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a845d7-0710-4f94-9b88-3a817cb25794 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:35.292687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:35.299850 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 505/1015] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Aug 18 14:39:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:35.302714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f122f9e-0afa-4557-bd84-e33c689c7dae None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:35.405967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f122f9e-0afa-4557-bd84-e33c689c7dae None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:35.419272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:35.440615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5347ff-686e-4722-a47f-dc3f61a2610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:35.441220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5347ff-686e-4722-a47f-dc3f61a2610d None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:35.441499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5347ff-686e-4722-a47f-dc3f61a2610d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:35.441771 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5347ff-686e-4722-a47f-dc3f61a2610d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:35.442432 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5347ff-686e-4722-a47f-dc3f61a2610d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:35.443822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5347ff-686e-4722-a47f-dc3f61a2610d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:35.475747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 511/1016] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:39:35 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:35.515409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0836bf-157b-46ec-a2d7-fcfc0dd36fc3 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:35.516017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0836bf-157b-46ec-a2d7-fcfc0dd36fc3 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:35.516017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0836bf-157b-46ec-a2d7-fcfc0dd36fc3 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:35.517346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 506/1017] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:35.523358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c54f0-2fe8-44ec-b138-1e801397e61a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:35.523732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c54f0-2fe8-44ec-b138-1e801397e61a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:35.524023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c54f0-2fe8-44ec-b138-1e801397e61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:35.543720 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a04c54f0-2fe8-44ec-b138-1e801397e61a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:35.544978 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a04c54f0-2fe8-44ec-b138-1e801397e61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:35.628287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 512/1018] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:35.895108 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0836bf-157b-46ec-a2d7-fcfc0dd36fc3 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:36.002102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0836bf-157b-46ec-a2d7-fcfc0dd36fc3 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:36.037149 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e2026-c49c-4601-913a-8482e844b998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:36.037782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e2026-c49c-4601-913a-8482e844b998 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:36.038062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e2026-c49c-4601-913a-8482e844b998 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:36.038336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e2026-c49c-4601-913a-8482e844b998 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:36.039460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e2026-c49c-4601-913a-8482e844b998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:36.040934 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e2026-c49c-4601-913a-8482e844b998 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:36.043721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 507/1019] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:36.049011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c54f0-2fe8-44ec-b138-1e801397e61a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:36.140603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:36.147727 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c54f0-2fe8-44ec-b138-1e801397e61a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:36.181853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877cea9-1496-41e5-a396-23e873cef1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:36.182457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877cea9-1496-41e5-a396-23e873cef1e6 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:36.182731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877cea9-1496-41e5-a396-23e873cef1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:36.182999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877cea9-1496-41e5-a396-23e873cef1e6 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:36.183643 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877cea9-1496-41e5-a396-23e873cef1e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:36.185081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877cea9-1496-41e5-a396-23e873cef1e6 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:36.211514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 513/1020] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:39:36 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:39:36.543860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c784d7-0168-4ead-a8f7-8c27344d235f None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:36.544402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c784d7-0168-4ead-a8f7-8c27344d235f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:36.544402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c784d7-0168-4ead-a8f7-8c27344d235f None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:36.546156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 508/1021] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:36.551504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725acfe5-a2de-4a1b-8591-d25de8014710 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:36.551844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725acfe5-a2de-4a1b-8591-d25de8014710 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:36.552121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725acfe5-a2de-4a1b-8591-d25de8014710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:36.570195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-725acfe5-a2de-4a1b-8591-d25de8014710 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:36.571405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-725acfe5-a2de-4a1b-8591-d25de8014710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:36.638111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 514/1022] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:36.904431 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c784d7-0168-4ead-a8f7-8c27344d235f None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:36.972943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:37.009968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c784d7-0168-4ead-a8f7-8c27344d235f None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:37.043191 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f888ba-28b4-45d5-8771-37cfdeadcfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:37.043861 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f888ba-28b4-45d5-8771-37cfdeadcfdb None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:37.044144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f888ba-28b4-45d5-8771-37cfdeadcfdb None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:37.044417 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f888ba-28b4-45d5-8771-37cfdeadcfdb None admin] PATH_INFO: `/v3/services/image` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:37.045000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f888ba-28b4-45d5-8771-37cfdeadcfdb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:37.046398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f888ba-28b4-45d5-8771-37cfdeadcfdb None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-d1f888ba-28b4-45d5-8771-37cfdeadcfdb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:37.055053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:37.056080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 14:39:37.056930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:37.059749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 509/1023] 10.222.0.36 () {56 vars in 1105 bytes} [Thu Aug 18 14:39:36 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:37.063733 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725acfe5-a2de-4a1b-8591-d25de8014710 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:37.146433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:37.171355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725acfe5-a2de-4a1b-8591-d25de8014710 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:37.210474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367525-8fb2-45e6-88ff-f7fd088227c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:37.211085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367525-8fb2-45e6-88ff-f7fd088227c1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:37.211363 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367525-8fb2-45e6-88ff-f7fd088227c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:37.211634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367525-8fb2-45e6-88ff-f7fd088227c1 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:37.212301 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50367525-8fb2-45e6-88ff-f7fd088227c1 None admin] RBAC: Adding query filter params (name=image) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:37.212606 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50367525-8fb2-45e6-88ff-f7fd088227c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:37.213977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50367525-8fb2-45e6-88ff-f7fd088227c1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:37.218990 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 515/1024] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:37.224832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f888ba-28b4-45d5-8771-37cfdeadcfdb None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:37.329730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f888ba-28b4-45d5-8771-37cfdeadcfdb None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:37.360302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:37.367193 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4553e3-7a8c-49ea-80fa-0a9f7307c0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:37.367828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4553e3-7a8c-49ea-80fa-0a9f7307c0dd None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:37.368112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4553e3-7a8c-49ea-80fa-0a9f7307c0dd None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:37.368385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4553e3-7a8c-49ea-80fa-0a9f7307c0dd None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:37.369013 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4553e3-7a8c-49ea-80fa-0a9f7307c0dd None admin] RBAC: Adding query filter params (type=image) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:37.369316 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4553e3-7a8c-49ea-80fa-0a9f7307c0dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:37.370681 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4553e3-7a8c-49ea-80fa-0a9f7307c0dd None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:37.376467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 510/1025] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:37.382351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367525-8fb2-45e6-88ff-f7fd088227c1 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:37.481544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367525-8fb2-45e6-88ff-f7fd088227c1 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:37.482201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dd8d7-1ec0-42aa-aef3-cbed93136e57 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:37.482768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dd8d7-1ec0-42aa-aef3-cbed93136e57 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:37.483979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dd8d7-1ec0-42aa-aef3-cbed93136e57 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:37.486673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 511/1026] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:37.491491 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4883f-f786-4404-90f8-489f8f72032d None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:37.491888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4883f-f786-4404-90f8-489f8f72032d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:37.492172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4883f-f786-4404-90f8-489f8f72032d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:37.511377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-0de4883f-f786-4404-90f8-489f8f72032d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:37.512642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0de4883f-f786-4404-90f8-489f8f72032d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:37.520810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2db02-2cca-421c-ad4f-5c36c98d4a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:37.521498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2db02-2cca-421c-ad4f-5c36c98d4a98 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:37.521776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2db02-2cca-421c-ad4f-5c36c98d4a98 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:37.522045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2db02-2cca-421c-ad4f-5c36c98d4a98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:37.522761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2db02-2cca-421c-ad4f-5c36c98d4a98 None admin] RBAC: Adding query filter params (interface=public, service_id=504205a371f44df1a8d5fadc140105f3, region_id=RegionOne) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:37.523070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2db02-2cca-421c-ad4f-5c36c98d4a98 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:37.524435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2db02-2cca-421c-ad4f-5c36c98d4a98 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:37.530083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 516/1027] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Aug 18 14:39:37 2022] GET /identity/v3/endpoints?service_id=504205a371f44df1a8d5fadc140105f3&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:37.535340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2db02-2cca-421c-ad4f-5c36c98d4a98 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:37.545062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:37.587182 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 512/1028] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:37.640470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2db02-2cca-421c-ad4f-5c36c98d4a98 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:37.675893 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19d02f-3bc6-4443-a56d-1f2be6d2fe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:37.676494 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19d02f-3bc6-4443-a56d-1f2be6d2fe79 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:37.676767 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19d02f-3bc6-4443-a56d-1f2be6d2fe79 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:37.677034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19d02f-3bc6-4443-a56d-1f2be6d2fe79 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:37.677627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19d02f-3bc6-4443-a56d-1f2be6d2fe79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:37.678982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19d02f-3bc6-4443-a56d-1f2be6d2fe79 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:37.691991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 517/1029] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Aug 18 14:39:37 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:39:37.854981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4883f-f786-4404-90f8-489f8f72032d None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:37.961166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4883f-f786-4404-90f8-489f8f72032d None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:37.985902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:37.997569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09ee41-4c39-4527-a4a6-a399252c4e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:37.998206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09ee41-4c39-4527-a4a6-a399252c4e68 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:37.998483 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09ee41-4c39-4527-a4a6-a399252c4e68 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:37.998768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09ee41-4c39-4527-a4a6-a399252c4e68 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:38.009391 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09ee41-4c39-4527-a4a6-a399252c4e68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-fb09ee41-4c39-4527-a4a6-a399252c4e68 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:38.014037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:38.015472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:38.020632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 513/1030] 10.222.0.36 () {56 vars in 1120 bytes} [Thu Aug 18 14:39:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:38.022222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19d02f-3bc6-4443-a56d-1f2be6d2fe79 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:38.093270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:38.125243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19d02f-3bc6-4443-a56d-1f2be6d2fe79 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:38.160109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731a60d-fb9d-456e-a7ea-426172ac6806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:38.160737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731a60d-fb9d-456e-a7ea-426172ac6806 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:38.160981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731a60d-fb9d-456e-a7ea-426172ac6806 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:38.161238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731a60d-fb9d-456e-a7ea-426172ac6806 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:38.161893 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731a60d-fb9d-456e-a7ea-426172ac6806 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:38.162191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731a60d-fb9d-456e-a7ea-426172ac6806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:38.163543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731a60d-fb9d-456e-a7ea-426172ac6806 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:38.166879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48925256-bbc2-4511-bca4-96d3de86e235 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:38.167165 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48925256-bbc2-4511-bca4-96d3de86e235 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:38.167437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48925256-bbc2-4511-bca4-96d3de86e235 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:38.168823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 514/1031] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:38.176540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 518/1032] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Aug 18 14:39:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:38.176684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81586ad7-57fc-4c56-bdff-a8981ce3327a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:38.176684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81586ad7-57fc-4c56-bdff-a8981ce3327a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:38.176684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81586ad7-57fc-4c56-bdff-a8981ce3327a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:38.177340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731a60d-fb9d-456e-a7ea-426172ac6806 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:38.193136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-81586ad7-57fc-4c56-bdff-a8981ce3327a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:38.194346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-81586ad7-57fc-4c56-bdff-a8981ce3327a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:38.267735 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 515/1033] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:38.282908 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731a60d-fb9d-456e-a7ea-426172ac6806 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:38.312238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:38.317625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd653d-8c94-4661-bb52-e917b2ac9263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:38.318469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd653d-8c94-4661-bb52-e917b2ac9263 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:38.318931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd653d-8c94-4661-bb52-e917b2ac9263 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:38.319403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd653d-8c94-4661-bb52-e917b2ac9263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:38.330899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd653d-8c94-4661-bb52-e917b2ac9263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:38.332575 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd653d-8c94-4661-bb52-e917b2ac9263 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-efcd653d-8c94-4661-bb52-e917b2ac9263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:38.347346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:38.348420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:38.349345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:38.349345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:38.349345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:38.349527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 519/1034] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:38.355611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81586ad7-57fc-4c56-bdff-a8981ce3327a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:38.458637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81586ad7-57fc-4c56-bdff-a8981ce3327a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:38.472518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:38.493169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89a168-9aaf-461f-a47c-f455f0eb679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:38.493757 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89a168-9aaf-461f-a47c-f455f0eb679e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:38.494036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89a168-9aaf-461f-a47c-f455f0eb679e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:38.494299 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89a168-9aaf-461f-a47c-f455f0eb679e None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:38.494926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89a168-9aaf-461f-a47c-f455f0eb679e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:38.495228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89a168-9aaf-461f-a47c-f455f0eb679e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:38.496608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89a168-9aaf-461f-a47c-f455f0eb679e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:38.509325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 516/1035] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:38.515313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd653d-8c94-4661-bb52-e917b2ac9263 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:38.538677 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89a168-9aaf-461f-a47c-f455f0eb679e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:38.624668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd653d-8c94-4661-bb52-e917b2ac9263 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:38.645420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89a168-9aaf-461f-a47c-f455f0eb679e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:38.661519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4e67d-e2b1-432a-b865-763c5d666714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:38.662164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4e67d-e2b1-432a-b865-763c5d666714 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:38.662408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4e67d-e2b1-432a-b865-763c5d666714 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:38.662680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4e67d-e2b1-432a-b865-763c5d666714 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:38.672209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4e67d-e2b1-432a-b865-763c5d666714 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:38.673985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4e67d-e2b1-432a-b865-763c5d666714 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:38.683911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1c042-c278-41d0-868f-43169b4605ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:38.684537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1c042-c278-41d0-868f-43169b4605ed None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:38.684814 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1c042-c278-41d0-868f-43169b4605ed None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:38.685083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1c042-c278-41d0-868f-43169b4605ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-22c4e67d-e2b1-432a-b865-763c5d666714 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:38.685372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:38.686322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:38.687316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:38.687316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:38.687316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:38.687316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:38.687316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:38.687316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:38.687316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:38.687316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:38.687316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:39:38.687316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:38.687316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 520/1036] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Aug 18 14:39:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:38.689101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1c042-c278-41d0-868f-43169b4605ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:38.690525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1c042-c278-41d0-868f-43169b4605ed None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:38.696055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4e67d-e2b1-432a-b865-763c5d666714 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:38.696197 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 517/1037] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:38.697677 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1c042-c278-41d0-868f-43169b4605ed None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:38.742003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:38.747178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:38.806103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4e67d-e2b1-432a-b865-763c5d666714 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:38.810045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1c042-c278-41d0-868f-43169b4605ed None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:38.842069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e1b2-bc65-47e5-b39f-c4eeebda12db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:38.843226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e1b2-bc65-47e5-b39f-c4eeebda12db None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:38.843498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e1b2-bc65-47e5-b39f-c4eeebda12db None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:38.843788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e1b2-bc65-47e5-b39f-c4eeebda12db None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:38.844534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958e1b2-bc65-47e5-b39f-c4eeebda12db None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:38.844834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958e1b2-bc65-47e5-b39f-c4eeebda12db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:38.845343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1f80b-009f-428a-824f-8bd4a7b27dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:38.845946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1f80b-009f-428a-824f-8bd4a7b27dd6 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:38.846225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1f80b-009f-428a-824f-8bd4a7b27dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:38.846321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958e1b2-bc65-47e5-b39f-c4eeebda12db None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:38.846499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1f80b-009f-428a-824f-8bd4a7b27dd6 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:38.847135 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1f80b-009f-428a-824f-8bd4a7b27dd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:38.848622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1f80b-009f-428a-824f-8bd4a7b27dd6 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:38.867746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 521/1038] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Aug 18 14:39:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:38.871485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e1b2-bc65-47e5-b39f-c4eeebda12db None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:38.875717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 518/1039] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:39:38 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:38.985945 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e1b2-bc65-47e5-b39f-c4eeebda12db None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:38.987716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:39.025112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0deb3-1768-4379-b8ae-877fab9564ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:39.025649 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0deb3-1768-4379-b8ae-877fab9564ab None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:39.026673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0deb3-1768-4379-b8ae-877fab9564ab None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:39.026673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0deb3-1768-4379-b8ae-877fab9564ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:39.095796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0deb3-1768-4379-b8ae-877fab9564ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:39.097255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0deb3-1768-4379-b8ae-877fab9564ab None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:39.194091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 522/1040] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:38 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 14:39:39.200792 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1f80b-009f-428a-824f-8bd4a7b27dd6 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:39.299565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:39.304128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1f80b-009f-428a-824f-8bd4a7b27dd6 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:39.338417 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d8334-cd32-4ca7-b4fb-1f02d2d1fab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:39.339041 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d8334-cd32-4ca7-b4fb-1f02d2d1fab1 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:39.339315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d8334-cd32-4ca7-b4fb-1f02d2d1fab1 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:39.339578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d8334-cd32-4ca7-b4fb-1f02d2d1fab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:39.351389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d8334-cd32-4ca7-b4fb-1f02d2d1fab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:39.353025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d8334-cd32-4ca7-b4fb-1f02d2d1fab1 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-f43d8334-cd32-4ca7-b4fb-1f02d2d1fab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:39.368058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:39.369043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:39.369979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:39.369979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:39.369979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:39.371784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 519/1041] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:39.375375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0deb3-1768-4379-b8ae-877fab9564ab None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:39.377896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:39.477364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0deb3-1768-4379-b8ae-877fab9564ab None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:39.510455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b6b49-c6a6-4e3c-9294-17243d701f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:39.511346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b6b49-c6a6-4e3c-9294-17243d701f64 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:39.511835 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b6b49-c6a6-4e3c-9294-17243d701f64 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:39.512308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b6b49-c6a6-4e3c-9294-17243d701f64 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:39.513115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b6b49-c6a6-4e3c-9294-17243d701f64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:39.513589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b6b49-c6a6-4e3c-9294-17243d701f64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:39.515642 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b6b49-c6a6-4e3c-9294-17243d701f64 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:39.531765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 523/1042] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:39.533549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d8334-cd32-4ca7-b4fb-1f02d2d1fab1 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:39.607275 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93717cc6-9316-45b3-804f-823914ff09bd None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:39.607461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93717cc6-9316-45b3-804f-823914ff09bd None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:39.607758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93717cc6-9316-45b3-804f-823914ff09bd None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:39.609213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 524/1043] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:39:39.614082 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051ffb8-6616-4a94-9b51-4e5599811784 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:39.614371 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051ffb8-6616-4a94-9b51-4e5599811784 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:39.614647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051ffb8-6616-4a94-9b51-4e5599811784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:39.632438 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-c051ffb8-6616-4a94-9b51-4e5599811784 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:39.633631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c051ffb8-6616-4a94-9b51-4e5599811784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:39.646457 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d8334-cd32-4ca7-b4fb-1f02d2d1fab1 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:39.683295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076c968-a0fd-4c37-bb8a-1f571b79f8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:39.683981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076c968-a0fd-4c37-bb8a-1f571b79f8bc None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:39.684300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076c968-a0fd-4c37-bb8a-1f571b79f8bc None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:39.684614 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076c968-a0fd-4c37-bb8a-1f571b79f8bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:39.691395 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076c968-a0fd-4c37-bb8a-1f571b79f8bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:39.692911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076c968-a0fd-4c37-bb8a-1f571b79f8bc None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:39.695583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-2076c968-a0fd-4c37-bb8a-1f571b79f8bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:39.706267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:39.707223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:39.708190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:39.708190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:39.708190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:39.708190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:39.708190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:39.708190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:39.708190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:39.708190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:39.708190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 520/1044] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:39.710522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 525/1045] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:39.713309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076c968-a0fd-4c37-bb8a-1f571b79f8bc None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:39.726878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:39.815097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076c968-a0fd-4c37-bb8a-1f571b79f8bc None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:39.848896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e77b9a-c172-4482-9e1b-bbfe71fe6d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:39.849498 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e77b9a-c172-4482-9e1b-bbfe71fe6d80 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:39.849844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e77b9a-c172-4482-9e1b-bbfe71fe6d80 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:39.850118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e77b9a-c172-4482-9e1b-bbfe71fe6d80 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:39.850798 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e77b9a-c172-4482-9e1b-bbfe71fe6d80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:39.851103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e77b9a-c172-4482-9e1b-bbfe71fe6d80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:39.852556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e77b9a-c172-4482-9e1b-bbfe71fe6d80 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:39.869458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 521/1046] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:39.871275 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051ffb8-6616-4a94-9b51-4e5599811784 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:39.976835 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051ffb8-6616-4a94-9b51-4e5599811784 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:39.977179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e77b9a-c172-4482-9e1b-bbfe71fe6d80 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:40.012607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa555da-dd80-4b3a-a0aa-fbd6a695eb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:40.013206 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa555da-dd80-4b3a-a0aa-fbd6a695eb90 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:40.013485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa555da-dd80-4b3a-a0aa-fbd6a695eb90 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:40.013757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa555da-dd80-4b3a-a0aa-fbd6a695eb90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:40.015039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa555da-dd80-4b3a-a0aa-fbd6a695eb90 None admin] RBAC: Adding query filter params (role.id=b5a4a4e7850147889e8dfd1344c75440, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=3f3efa9f3eb24864943b68d241435817) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:40.015348 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa555da-dd80-4b3a-a0aa-fbd6a695eb90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:40.016811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa555da-dd80-4b3a-a0aa-fbd6a695eb90 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:40.022404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 526/1047] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:39:39 2022] GET /identity/v3/role_assignments?user.id=3f3efa9f3eb24864943b68d241435817&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=b5a4a4e7850147889e8dfd1344c75440 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:40.056337 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:40.092520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e77b9a-c172-4482-9e1b-bbfe71fe6d80 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:40.131397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529ab29-5aec-48c2-a7b7-4c2d3957b178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:40.132006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529ab29-5aec-48c2-a7b7-4c2d3957b178 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:40.132283 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529ab29-5aec-48c2-a7b7-4c2d3957b178 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:40.132551 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529ab29-5aec-48c2-a7b7-4c2d3957b178 None admin] PATH_INFO: `/v3/services/image` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:40.133139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529ab29-5aec-48c2-a7b7-4c2d3957b178 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:40.134535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529ab29-5aec-48c2-a7b7-4c2d3957b178 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-3529ab29-5aec-48c2-a7b7-4c2d3957b178 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:40.144832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:40.145865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:40.148713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 522/1048] 10.222.0.36 () {56 vars in 1105 bytes} [Thu Aug 18 14:39:39 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:40.152325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa555da-dd80-4b3a-a0aa-fbd6a695eb90 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:40.210606 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:40.256115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa555da-dd80-4b3a-a0aa-fbd6a695eb90 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:40.296729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bc9e3-5083-43c1-acfa-d10fbde159b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:40.297355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bc9e3-5083-43c1-acfa-d10fbde159b8 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:40.297633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bc9e3-5083-43c1-acfa-d10fbde159b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:40.297901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bc9e3-5083-43c1-acfa-d10fbde159b8 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:40.298534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bc9e3-5083-43c1-acfa-d10fbde159b8 None admin] RBAC: Adding query filter params (name=image) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:40.298836 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bc9e3-5083-43c1-acfa-d10fbde159b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:40.300233 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bc9e3-5083-43c1-acfa-d10fbde159b8 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:40.310076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 527/1049] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:40.312036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529ab29-5aec-48c2-a7b7-4c2d3957b178 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:40.414703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529ab29-5aec-48c2-a7b7-4c2d3957b178 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:40.448894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0be92c-7356-46f5-b17e-510f4536347b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:40.449793 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0be92c-7356-46f5-b17e-510f4536347b None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:40.450250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0be92c-7356-46f5-b17e-510f4536347b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:40.450731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0be92c-7356-46f5-b17e-510f4536347b None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:40.451504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0be92c-7356-46f5-b17e-510f4536347b None admin] RBAC: Adding query filter params (type=image) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:40.452024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0be92c-7356-46f5-b17e-510f4536347b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:40.453730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0be92c-7356-46f5-b17e-510f4536347b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:40.455252 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:40.465032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 523/1050] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:40.466331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bc9e3-5083-43c1-acfa-d10fbde159b8 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:40.566389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bc9e3-5083-43c1-acfa-d10fbde159b8 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:40.585434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:40.600865 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c69b11-1083-4fca-9eb5-143244548cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:40.601472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c69b11-1083-4fca-9eb5-143244548cb9 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:40.601746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c69b11-1083-4fca-9eb5-143244548cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:40.602013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c69b11-1083-4fca-9eb5-143244548cb9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:40.602673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c69b11-1083-4fca-9eb5-143244548cb9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:40.604060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c69b11-1083-4fca-9eb5-143244548cb9 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:40.631914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 528/1051] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:39:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:40.896457 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce61628-c07c-4e16-a869-5418b1c8160c None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:40.896972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce61628-c07c-4e16-a869-5418b1c8160c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:40.897049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce61628-c07c-4e16-a869-5418b1c8160c None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:40.898382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 524/1052] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:40.903617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ca19a-c77d-4491-ba4a-ff72c5e94216 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:40.903942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ca19a-c77d-4491-ba4a-ff72c5e94216 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:40.904213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ca19a-c77d-4491-ba4a-ff72c5e94216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:40.924431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-e07ca19a-c77d-4491-ba4a-ff72c5e94216 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:40.925634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e07ca19a-c77d-4491-ba4a-ff72c5e94216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:41.000536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 529/1053] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:41.263339 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce61628-c07c-4e16-a869-5418b1c8160c None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:41.367071 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce61628-c07c-4e16-a869-5418b1c8160c None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:41.400933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bfb61-a485-4dec-9cd6-efc3405c61e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:41.401552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bfb61-a485-4dec-9cd6-efc3405c61e8 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:41.401829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bfb61-a485-4dec-9cd6-efc3405c61e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:41.402096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bfb61-a485-4dec-9cd6-efc3405c61e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:41.412759 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0bfb61-a485-4dec-9cd6-efc3405c61e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-fd0bfb61-a485-4dec-9cd6-efc3405c61e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:41.417411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:41.418341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:41.419214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:41.419214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:41.419214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:41.419214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:41.419214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:41.419214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:41.419214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:41.419214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:41.419214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:41.419214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:41.419214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:41.419214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:41.423658 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 525/1054] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:39:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:41.424936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ca19a-c77d-4491-ba4a-ff72c5e94216 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:41.525068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ca19a-c77d-4491-ba4a-ff72c5e94216 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:41.526548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:41.567564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e6d5d-f58a-4b18-97ee-fdcfb9060b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:41.568206 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e6d5d-f58a-4b18-97ee-fdcfb9060b80 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:41.568479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e6d5d-f58a-4b18-97ee-fdcfb9060b80 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:41.568746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e6d5d-f58a-4b18-97ee-fdcfb9060b80 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:41.569408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e6d5d-f58a-4b18-97ee-fdcfb9060b80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:41.569710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e6d5d-f58a-4b18-97ee-fdcfb9060b80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:41.571064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e6d5d-f58a-4b18-97ee-fdcfb9060b80 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:41.584810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 530/1055] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:39:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:41.586114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bfb61-a485-4dec-9cd6-efc3405c61e8 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:41.647743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:41.690926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bfb61-a485-4dec-9cd6-efc3405c61e8 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:41.726763 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78f93e-b09b-4dd6-84ba-891a4fe4348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:41.727398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78f93e-b09b-4dd6-84ba-891a4fe4348d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:41.727706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78f93e-b09b-4dd6-84ba-891a4fe4348d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:41.727978 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78f93e-b09b-4dd6-84ba-891a4fe4348d None admin] PATH_INFO: `/v3/users/8495719c77c84988b1255e7670dee0c0` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:41.742001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78f93e-b09b-4dd6-84ba-891a4fe4348d None admin] RBAC: Authorizing `identity:get_user(user_id=8495719c77c84988b1255e7670dee0c0)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:41.743438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78f93e-b09b-4dd6-84ba-891a4fe4348d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:41.745242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 526/1056] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:39:41 2022] GET /identity/v3/users/8495719c77c84988b1255e7670dee0c0 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:41.752970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e6d5d-f58a-4b18-97ee-fdcfb9060b80 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:41.852589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e6d5d-f58a-4b18-97ee-fdcfb9060b80 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:41.885967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eefb97-20a0-4131-b500-4fe41514ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:41.886555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eefb97-20a0-4131-b500-4fe41514ed7f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:41.886831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eefb97-20a0-4131-b500-4fe41514ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:41.887099 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eefb97-20a0-4131-b500-4fe41514ed7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:41.913127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e8769-161d-42bc-9490-e022fd775ed4 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:41.913638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e8769-161d-42bc-9490-e022fd775ed4 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:41.913739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e8769-161d-42bc-9490-e022fd775ed4 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:41.916179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 527/1057] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:41.921215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87796a14-f155-415a-b83f-9f9fb368c93d None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:41.921495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87796a14-f155-415a-b83f-9f9fb368c93d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:41.921766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87796a14-f155-415a-b83f-9f9fb368c93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:41.925410 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:41.942155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-87796a14-f155-415a-b83f-9f9fb368c93d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:41.943550 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-87796a14-f155-415a-b83f-9f9fb368c93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:41.963805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eefb97-20a0-4131-b500-4fe41514ed7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:41.965365 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eefb97-20a0-4131-b500-4fe41514ed7f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:42.011155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 528/1058] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:42.070743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 531/1059] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:39:42.071964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87796a14-f155-415a-b83f-9f9fb368c93d None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:42.119201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:42.173082 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87796a14-f155-415a-b83f-9f9fb368c93d None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:42.207128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d4a60-d38c-48c4-8c07-8af17e602e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:42.207775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d4a60-d38c-48c4-8c07-8af17e602e27 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:42.208048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d4a60-d38c-48c4-8c07-8af17e602e27 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:42.208312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d4a60-d38c-48c4-8c07-8af17e602e27 None admin] PATH_INFO: `/v3/projects/b2c2806854644a1eabfa5eefef46881b` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:42.209416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d4a60-d38c-48c4-8c07-8af17e602e27 None admin] RBAC: Authorizing `identity:get_project(project_id=b2c2806854644a1eabfa5eefef46881b)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:42.210832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d4a60-d38c-48c4-8c07-8af17e602e27 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:42.218005 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 529/1060] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:39:42 2022] GET /identity/v3/projects/b2c2806854644a1eabfa5eefef46881b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:42.218952 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eefb97-20a0-4131-b500-4fe41514ed7f None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:42.278520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d4a60-d38c-48c4-8c07-8af17e602e27 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:42.279089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:42.319814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eefb97-20a0-4131-b500-4fe41514ed7f None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:42.356452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01b680-f1b4-4372-82c5-bcbcbba12ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:42.357058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01b680-f1b4-4372-82c5-bcbcbba12ef2 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:42.357335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01b680-f1b4-4372-82c5-bcbcbba12ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:42.357662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01b680-f1b4-4372-82c5-bcbcbba12ef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:42.358947 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01b680-f1b4-4372-82c5-bcbcbba12ef2 None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=b2c2806854644a1eabfa5eefef46881b, user.id=8495719c77c84988b1255e7670dee0c0) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:42.359252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01b680-f1b4-4372-82c5-bcbcbba12ef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:42.360705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01b680-f1b4-4372-82c5-bcbcbba12ef2 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:42.366307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 532/1061] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:39:42 2022] GET /identity/v3/role_assignments?user.id=8495719c77c84988b1255e7670dee0c0&scope.project.id=b2c2806854644a1eabfa5eefef46881b&role.id=9a439cf8eef34ada85428b378878e63e => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:42.390926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d4a60-d38c-48c4-8c07-8af17e602e27 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:42.407290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:42.431883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea55e5-3c9e-458b-adb0-4021992f4149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:42.432535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea55e5-3c9e-458b-adb0-4021992f4149 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:42.432809 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea55e5-3c9e-458b-adb0-4021992f4149 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:42.433073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea55e5-3c9e-458b-adb0-4021992f4149 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:42.445176 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea55e5-3c9e-458b-adb0-4021992f4149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-3eea55e5-3c9e-458b-adb0-4021992f4149 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:42.449931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:42.450880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:42.453156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 530/1062] 10.222.0.36 () {56 vars in 1120 bytes} [Thu Aug 18 14:39:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:42.457997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01b680-f1b4-4372-82c5-bcbcbba12ef2 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:42.543342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc04be-5384-4a7c-8a9b-b45027dc36de None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:42.543911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc04be-5384-4a7c-8a9b-b45027dc36de None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:42.544425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc04be-5384-4a7c-8a9b-b45027dc36de None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:42.545994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 531/1063] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:42.550748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef49dfc-f0a5-4f1f-9c5f-db4cc4429e52 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:42.551094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef49dfc-f0a5-4f1f-9c5f-db4cc4429e52 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:42.551430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef49dfc-f0a5-4f1f-9c5f-db4cc4429e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:42.565781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01b680-f1b4-4372-82c5-bcbcbba12ef2 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:42.572646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4ef49dfc-f0a5-4f1f-9c5f-db4cc4429e52 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:42.573876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4ef49dfc-f0a5-4f1f-9c5f-db4cc4429e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:42.593616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:42.603166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809bf29-484b-4db9-8d0e-9b596322eb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:42.603798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809bf29-484b-4db9-8d0e-9b596322eb5b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:42.604075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809bf29-484b-4db9-8d0e-9b596322eb5b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:42.604343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809bf29-484b-4db9-8d0e-9b596322eb5b None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:42.604996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809bf29-484b-4db9-8d0e-9b596322eb5b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:42.605295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809bf29-484b-4db9-8d0e-9b596322eb5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:42.606647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809bf29-484b-4db9-8d0e-9b596322eb5b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:42.619408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 533/1064] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Aug 18 14:39:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:42.620478 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809bf29-484b-4db9-8d0e-9b596322eb5b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:42.650180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 532/1065] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:42.726514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809bf29-484b-4db9-8d0e-9b596322eb5b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:42.761607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab016c-4eac-4d50-a86b-36a132333971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:42.762257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab016c-4eac-4d50-a86b-36a132333971 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:42.762590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab016c-4eac-4d50-a86b-36a132333971 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:42.762773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab016c-4eac-4d50-a86b-36a132333971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:42.773921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab016c-4eac-4d50-a86b-36a132333971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:42.775358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab016c-4eac-4d50-a86b-36a132333971 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-11ab016c-4eac-4d50-a86b-36a132333971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:42.789741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:42.790685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:42.796635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:42.796635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:42.796635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:42.796635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 534/1066] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:42.796875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef49dfc-f0a5-4f1f-9c5f-db4cc4429e52 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:42.900370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef49dfc-f0a5-4f1f-9c5f-db4cc4429e52 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:42.903521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:42.917769 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab016c-4eac-4d50-a86b-36a132333971 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:42.949449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3e347-4ddf-4a88-9c24-a890a3daa404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:42.951189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3e347-4ddf-4a88-9c24-a890a3daa404 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:42.951651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3e347-4ddf-4a88-9c24-a890a3daa404 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:42.952143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3e347-4ddf-4a88-9c24-a890a3daa404 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:42.952935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3e347-4ddf-4a88-9c24-a890a3daa404 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:42.953540 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3e347-4ddf-4a88-9c24-a890a3daa404 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:42.955877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3e347-4ddf-4a88-9c24-a890a3daa404 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:42.962787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:42.973558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 533/1067] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:42.979522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3e347-4ddf-4a88-9c24-a890a3daa404 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:43.031835 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab016c-4eac-4d50-a86b-36a132333971 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:43.067202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af9e0e-1c9b-4fc3-b976-a2c060c313ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:43.067834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af9e0e-1c9b-4fc3-b976-a2c060c313ee None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:43.068116 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af9e0e-1c9b-4fc3-b976-a2c060c313ee None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:43.068394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af9e0e-1c9b-4fc3-b976-a2c060c313ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:43.087820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3e347-4ddf-4a88-9c24-a890a3daa404 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:43.122972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5e75-3aa1-4b13-b213-8ecfc4ff9e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:43.123614 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5e75-3aa1-4b13-b213-8ecfc4ff9e08 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:43.123934 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5e75-3aa1-4b13-b213-8ecfc4ff9e08 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:43.124210 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5e75-3aa1-4b13-b213-8ecfc4ff9e08 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:43.131704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5e75-3aa1-4b13-b213-8ecfc4ff9e08 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:43.133424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5e75-3aa1-4b13-b213-8ecfc4ff9e08 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:43.136833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af9e0e-1c9b-4fc3-b976-a2c060c313ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:43.138276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af9e0e-1c9b-4fc3-b976-a2c060c313ee None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-4f6a5e75-3aa1-4b13-b213-8ecfc4ff9e08 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:43.144322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:43.145273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:43.146218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:43.146218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:43.146218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:43.146218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:43.146218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:43.146218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:43.146218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:43.146218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:43.146218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:39:43.146218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:43.150154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 534/1068] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Aug 18 14:39:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:43.151002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5e75-3aa1-4b13-b213-8ecfc4ff9e08 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:43.176729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:43.183228 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:43.242694 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 535/1069] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 14:39:43.248333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af9e0e-1c9b-4fc3-b976-a2c060c313ee None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:43.260579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5e75-3aa1-4b13-b213-8ecfc4ff9e08 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:43.296517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafdf60-f4c3-496c-a25c-06c82a0d6e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:43.297130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafdf60-f4c3-496c-a25c-06c82a0d6e02 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:43.297413 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafdf60-f4c3-496c-a25c-06c82a0d6e02 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:43.297685 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafdf60-f4c3-496c-a25c-06c82a0d6e02 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:43.298417 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafdf60-f4c3-496c-a25c-06c82a0d6e02 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:43.298723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafdf60-f4c3-496c-a25c-06c82a0d6e02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:43.300193 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafdf60-f4c3-496c-a25c-06c82a0d6e02 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:43.318833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 535/1070] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Aug 18 14:39:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:43.324733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafdf60-f4c3-496c-a25c-06c82a0d6e02 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:43.363357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af9e0e-1c9b-4fc3-b976-a2c060c313ee None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:43.399083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5b3ce-aa54-4fab-821c-8488c923588c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:43.399748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5b3ce-aa54-4fab-821c-8488c923588c None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:43.400033 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5b3ce-aa54-4fab-821c-8488c923588c None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:43.400307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5b3ce-aa54-4fab-821c-8488c923588c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:43.411096 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5b3ce-aa54-4fab-821c-8488c923588c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:43.411941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:43.412502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5b3ce-aa54-4fab-821c-8488c923588c None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:43.414289 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-a2e5b3ce-aa54-4fab-821c-8488c923588c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:43.430869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:43.431985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:43.432876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:43.432876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:43.432876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:43.432876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 536/1071] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:43.437598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafdf60-f4c3-496c-a25c-06c82a0d6e02 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:43.437888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5b3ce-aa54-4fab-821c-8488c923588c None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:43.476563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ca990-6f1a-4b83-bb86-a294d3915a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:43.477193 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ca990-6f1a-4b83-bb86-a294d3915a9c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:43.477470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ca990-6f1a-4b83-bb86-a294d3915a9c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:43.477742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ca990-6f1a-4b83-bb86-a294d3915a9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:43.488421 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ca990-6f1a-4b83-bb86-a294d3915a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:43.489866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ca990-6f1a-4b83-bb86-a294d3915a9c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-c28ca990-6f1a-4b83-bb86-a294d3915a9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:43.504427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:43.505385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:43.506352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:43.506352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:43.506352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:43.510366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 536/1072] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:43.511203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ca990-6f1a-4b83-bb86-a294d3915a9c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:43.544535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5b3ce-aa54-4fab-821c-8488c923588c None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:43.586687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a0707-225f-42db-9fd6-cc1fc21d4b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:43.587759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a0707-225f-42db-9fd6-cc1fc21d4b15 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:43.587759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a0707-225f-42db-9fd6-cc1fc21d4b15 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:43.587955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a0707-225f-42db-9fd6-cc1fc21d4b15 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:43.588493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a0707-225f-42db-9fd6-cc1fc21d4b15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:43.588798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a0707-225f-42db-9fd6-cc1fc21d4b15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:43.590744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a0707-225f-42db-9fd6-cc1fc21d4b15 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:43.611897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 537/1073] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:43.619918 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ca990-6f1a-4b83-bb86-a294d3915a9c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:43.659424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd2c8a-a43b-452f-8114-adb377ee7c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:43.660097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd2c8a-a43b-452f-8114-adb377ee7c9c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:43.660371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd2c8a-a43b-452f-8114-adb377ee7c9c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:43.660637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd2c8a-a43b-452f-8114-adb377ee7c9c None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:43.661345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd2c8a-a43b-452f-8114-adb377ee7c9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:43.661590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd2c8a-a43b-452f-8114-adb377ee7c9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:43.662986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd2c8a-a43b-452f-8114-adb377ee7c9c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:43.664989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:43.677401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 537/1074] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:43.683279 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a0707-225f-42db-9fd6-cc1fc21d4b15 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:43.748395 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:43.796299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a0707-225f-42db-9fd6-cc1fc21d4b15 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:43.837641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24d992-8c43-4710-99d2-b5b4ceab2353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:43.838284 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24d992-8c43-4710-99d2-b5b4ceab2353 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:43.838618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24d992-8c43-4710-99d2-b5b4ceab2353 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:43.838840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24d992-8c43-4710-99d2-b5b4ceab2353 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:43.845931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24d992-8c43-4710-99d2-b5b4ceab2353 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:43.847514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24d992-8c43-4710-99d2-b5b4ceab2353 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-3d24d992-8c43-4710-99d2-b5b4ceab2353 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:43.860618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:43.861701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:43.862916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:43.862916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:43.862916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:43.862916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:43.862916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:43.862916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:43.862916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:43.862916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:43.862916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 538/1075] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:43.868689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd2c8a-a43b-452f-8114-adb377ee7c9c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:43.976318 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd2c8a-a43b-452f-8114-adb377ee7c9c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:44.009732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2c362-ad65-42e2-9895-bf2dce6b030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:44.010330 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2c362-ad65-42e2-9895-bf2dce6b030f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:44.010605 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2c362-ad65-42e2-9895-bf2dce6b030f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:44.010890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2c362-ad65-42e2-9895-bf2dce6b030f None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:44.011550 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2c362-ad65-42e2-9895-bf2dce6b030f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:44.011885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2c362-ad65-42e2-9895-bf2dce6b030f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:44.013302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2c362-ad65-42e2-9895-bf2dce6b030f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:44.024455 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 538/1076] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:44.030454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24d992-8c43-4710-99d2-b5b4ceab2353 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:44.130824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24d992-8c43-4710-99d2-b5b4ceab2353 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:44.144770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:44.164758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc67b3-254b-473e-af54-a83662fdfa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:44.165407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc67b3-254b-473e-af54-a83662fdfa96 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:44.165681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc67b3-254b-473e-af54-a83662fdfa96 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:44.165952 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc67b3-254b-473e-af54-a83662fdfa96 None admin] PATH_INFO: `/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/3f3efa9f3eb24864943b68d241435817/roles/b5a4a4e7850147889e8dfd1344c75440` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:44.168094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc67b3-254b-473e-af54-a83662fdfa96 None admin] RBAC: Authorizing `identity:create_grant(project_id=12f9875f45a84e92a848b8fc1636d6c6, user_id=3f3efa9f3eb24864943b68d241435817, role_id=b5a4a4e7850147889e8dfd1344c75440)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:44.169515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc67b3-254b-473e-af54-a83662fdfa96 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:44.183256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 539/1077] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:39:44 2022] PUT /identity/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/3f3efa9f3eb24864943b68d241435817/roles/b5a4a4e7850147889e8dfd1344c75440 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:39:44.260700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb22de4-4c64-42ec-b330-ca0d50b40f19 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:44.261008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb22de4-4c64-42ec-b330-ca0d50b40f19 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:44.261351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb22de4-4c64-42ec-b330-ca0d50b40f19 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:44.262786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 539/1078] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:44.268618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b57b7-7cd3-494b-aa34-6c813d640a30 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:44.268887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b57b7-7cd3-494b-aa34-6c813d640a30 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:44.269160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b57b7-7cd3-494b-aa34-6c813d640a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:44.287734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-3e8b57b7-7cd3-494b-aa34-6c813d640a30 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:44.289144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3e8b57b7-7cd3-494b-aa34-6c813d640a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:44.360256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 540/1079] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:44.629010 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb22de4-4c64-42ec-b330-ca0d50b40f19 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:44.685520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:44.732598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb22de4-4c64-42ec-b330-ca0d50b40f19 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:44.766526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791d7c8-d07f-4958-b080-a7dd9b5477eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:44.767149 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791d7c8-d07f-4958-b080-a7dd9b5477eb None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:44.767428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791d7c8-d07f-4958-b080-a7dd9b5477eb None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:44.767730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791d7c8-d07f-4958-b080-a7dd9b5477eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:44.779175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791d7c8-d07f-4958-b080-a7dd9b5477eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-0791d7c8-d07f-4958-b080-a7dd9b5477eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:44.784470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:44.785564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:44.786791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 540/1080] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:39:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:44.791531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b57b7-7cd3-494b-aa34-6c813d640a30 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:44.886698 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b57b7-7cd3-494b-aa34-6c813d640a30 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:44.888898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:44.921620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8cf1e-1650-459a-8169-19c40861e9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:44.922223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8cf1e-1650-459a-8169-19c40861e9e5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:44.922504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8cf1e-1650-459a-8169-19c40861e9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:44.922776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8cf1e-1650-459a-8169-19c40861e9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:44.923434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8cf1e-1650-459a-8169-19c40861e9e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:44.923773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8cf1e-1650-459a-8169-19c40861e9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:44.925136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8cf1e-1650-459a-8169-19c40861e9e5 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:44.933114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 541/1081] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:39:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:44.938886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791d7c8-d07f-4958-b080-a7dd9b5477eb None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:45.036044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791d7c8-d07f-4958-b080-a7dd9b5477eb None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:45.049135 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:45.068674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef8fcf-c97f-4b35-b6d7-abe33ec2a5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:45.069291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef8fcf-c97f-4b35-b6d7-abe33ec2a5c5 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:45.069567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef8fcf-c97f-4b35-b6d7-abe33ec2a5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:45.069833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef8fcf-c97f-4b35-b6d7-abe33ec2a5c5 None admin] PATH_INFO: `/v3/users/8495719c77c84988b1255e7670dee0c0` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:45.070947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef8fcf-c97f-4b35-b6d7-abe33ec2a5c5 None admin] RBAC: Authorizing `identity:get_user(user_id=8495719c77c84988b1255e7670dee0c0)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:45.072394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef8fcf-c97f-4b35-b6d7-abe33ec2a5c5 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:45.073967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 541/1082] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:39:44 2022] GET /identity/v3/users/8495719c77c84988b1255e7670dee0c0 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:45.080245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8cf1e-1650-459a-8169-19c40861e9e5 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:45.177665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8cf1e-1650-459a-8169-19c40861e9e5 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:45.210975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0b142-3cb3-484f-adaf-bc5264e221f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:45.211601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0b142-3cb3-484f-adaf-bc5264e221f1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:45.211925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0b142-3cb3-484f-adaf-bc5264e221f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:45.212199 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0b142-3cb3-484f-adaf-bc5264e221f1 None admin] PATH_INFO: `/v3/projects/b2c2806854644a1eabfa5eefef46881b` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:45.213318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0b142-3cb3-484f-adaf-bc5264e221f1 None admin] RBAC: Authorizing `identity:get_project(project_id=b2c2806854644a1eabfa5eefef46881b)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:45.214757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0b142-3cb3-484f-adaf-bc5264e221f1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:45.216495 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 542/1083] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:39:45 2022] GET /identity/v3/projects/b2c2806854644a1eabfa5eefef46881b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:45.222678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef8fcf-c97f-4b35-b6d7-abe33ec2a5c5 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:45.320565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef8fcf-c97f-4b35-b6d7-abe33ec2a5c5 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:45.355778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf105b-aadc-4dff-8d86-0c4ec9de72e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:45.356428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf105b-aadc-4dff-8d86-0c4ec9de72e4 None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:45.356708 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf105b-aadc-4dff-8d86-0c4ec9de72e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:45.356983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf105b-aadc-4dff-8d86-0c4ec9de72e4 None admin] PATH_INFO: `/v3/projects/b2c2806854644a1eabfa5eefef46881b/users/8495719c77c84988b1255e7670dee0c0/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:45.358901 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf105b-aadc-4dff-8d86-0c4ec9de72e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2c2806854644a1eabfa5eefef46881b, user_id=8495719c77c84988b1255e7670dee0c0, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:45.360378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf105b-aadc-4dff-8d86-0c4ec9de72e4 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:45.372227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 542/1084] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:39:45 2022] PUT /identity/v3/projects/b2c2806854644a1eabfa5eefef46881b/users/8495719c77c84988b1255e7670dee0c0/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:39:46.055383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c9f2b-72db-4672-adf4-48de044bc9c8 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:46.056099 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c9f2b-72db-4672-adf4-48de044bc9c8 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:46.056099 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c9f2b-72db-4672-adf4-48de044bc9c8 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:46.057259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 543/1085] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:46.062433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96c36c-5e6b-4061-b373-cfff396043da None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:46.062850 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96c36c-5e6b-4061-b373-cfff396043da None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:46.062979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96c36c-5e6b-4061-b373-cfff396043da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:46.080601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-fc96c36c-5e6b-4061-b373-cfff396043da None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:46.081781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-fc96c36c-5e6b-4061-b373-cfff396043da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:46.116038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:46.147198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 543/1086] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:46.416376 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c9f2b-72db-4672-adf4-48de044bc9c8 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:46.453507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:46.518358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c9f2b-72db-4672-adf4-48de044bc9c8 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:46.550971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99cb06-34bb-49e8-b66d-a9d5afb1d937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:46.551583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99cb06-34bb-49e8-b66d-a9d5afb1d937 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:46.551884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99cb06-34bb-49e8-b66d-a9d5afb1d937 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:46.552153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99cb06-34bb-49e8-b66d-a9d5afb1d937 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:46.561826 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99cb06-34bb-49e8-b66d-a9d5afb1d937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-ea99cb06-34bb-49e8-b66d-a9d5afb1d937 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:46.566448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:46.567547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:46.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 544/1087] 10.222.0.36 () {56 vars in 1120 bytes} [Thu Aug 18 14:39:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:46.573445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96c36c-5e6b-4061-b373-cfff396043da None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:46.667770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96c36c-5e6b-4061-b373-cfff396043da None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:46.700503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e8625-b0a4-4f56-bbd6-af4cd78447c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:46.701098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e8625-b0a4-4f56-bbd6-af4cd78447c3 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:46.701368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e8625-b0a4-4f56-bbd6-af4cd78447c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:46.701632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e8625-b0a4-4f56-bbd6-af4cd78447c3 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:46.702826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e8625-b0a4-4f56-bbd6-af4cd78447c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:46.703123 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e8625-b0a4-4f56-bbd6-af4cd78447c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:46.704503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e8625-b0a4-4f56-bbd6-af4cd78447c3 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:46.712494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 544/1088] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Aug 18 14:39:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:46.718028 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99cb06-34bb-49e8-b66d-a9d5afb1d937 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:46.808449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:46.816166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99cb06-34bb-49e8-b66d-a9d5afb1d937 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:46.849033 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5c7ab-9fbe-4b31-947d-1710fa380b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:46.849651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5c7ab-9fbe-4b31-947d-1710fa380b4f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:46.849923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5c7ab-9fbe-4b31-947d-1710fa380b4f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:46.850190 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5c7ab-9fbe-4b31-947d-1710fa380b4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:46.859846 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5c7ab-9fbe-4b31-947d-1710fa380b4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:46.861273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5c7ab-9fbe-4b31-947d-1710fa380b4f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-73d5c7ab-9fbe-4b31-947d-1710fa380b4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:46.875991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:46.876938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:46.877822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:46.877822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:46.877822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:46.877822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 545/1089] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:46.882977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e8625-b0a4-4f56-bbd6-af4cd78447c3 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:46.920207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:46.988131 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e8625-b0a4-4f56-bbd6-af4cd78447c3 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:47.021221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970f2e4-d5d5-4fb4-8598-110b8e0fe8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:47.021822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970f2e4-d5d5-4fb4-8598-110b8e0fe8ff None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:47.022088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970f2e4-d5d5-4fb4-8598-110b8e0fe8ff None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:47.022359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970f2e4-d5d5-4fb4-8598-110b8e0fe8ff None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:47.022994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970f2e4-d5d5-4fb4-8598-110b8e0fe8ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:47.023306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970f2e4-d5d5-4fb4-8598-110b8e0fe8ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:47.024689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970f2e4-d5d5-4fb4-8598-110b8e0fe8ff None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:47.036433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 545/1090] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:47.041855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5c7ab-9fbe-4b31-947d-1710fa380b4f None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:47.138240 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5c7ab-9fbe-4b31-947d-1710fa380b4f None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:47.169643 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10102c87-a296-4917-9d60-498144a60d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:47.170309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10102c87-a296-4917-9d60-498144a60d70 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:47.171060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10102c87-a296-4917-9d60-498144a60d70 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:47.171326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10102c87-a296-4917-9d60-498144a60d70 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:47.178450 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10102c87-a296-4917-9d60-498144a60d70 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:47.179898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10102c87-a296-4917-9d60-498144a60d70 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-10102c87-a296-4917-9d60-498144a60d70 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:39:47.191059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:47.192185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:47.193368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:47.193368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:47.193368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:39:47.193368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:39:47.193368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:39:47.193368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:39:47.193368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:39:47.193368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:39:47.193368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 14:39:47.193368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:47.195967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 546/1091] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Aug 18 14:39:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:39:47.197795 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970f2e4-d5d5-4fb4-8598-110b8e0fe8ff None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:47.263269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f7ccb-fe4e-44d8-8f3f-39fed234df78 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:47.263551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f7ccb-fe4e-44d8-8f3f-39fed234df78 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:47.263910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f7ccb-fe4e-44d8-8f3f-39fed234df78 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:47.265399 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 547/1092] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:39:47.270029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28873d-c82e-4f6c-a48f-0faa834e7856 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:47.270310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28873d-c82e-4f6c-a48f-0faa834e7856 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:47.270584 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28873d-c82e-4f6c-a48f-0faa834e7856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:47.284006 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:47.289282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-0a28873d-c82e-4f6c-a48f-0faa834e7856 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:47.290485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-0a28873d-c82e-4f6c-a48f-0faa834e7856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:47.302367 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:47.309354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970f2e4-d5d5-4fb4-8598-110b8e0fe8ff None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:47.344468 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96756a00-6fbc-4072-aa56-64ac8abac13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:47.345190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96756a00-6fbc-4072-aa56-64ac8abac13f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:47.345552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96756a00-6fbc-4072-aa56-64ac8abac13f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:47.345893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96756a00-6fbc-4072-aa56-64ac8abac13f None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:47.346618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96756a00-6fbc-4072-aa56-64ac8abac13f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:47.347055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96756a00-6fbc-4072-aa56-64ac8abac13f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:47.348370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96756a00-6fbc-4072-aa56-64ac8abac13f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:47.370646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 546/1093] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Aug 18 14:39:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:47.370830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 548/1094] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:47.372368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96756a00-6fbc-4072-aa56-64ac8abac13f None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:47.469726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96756a00-6fbc-4072-aa56-64ac8abac13f None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:47.506748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad00415-28e7-4db7-a984-7a0973b81658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:47.507356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad00415-28e7-4db7-a984-7a0973b81658 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:47.507629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad00415-28e7-4db7-a984-7a0973b81658 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:47.507919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad00415-28e7-4db7-a984-7a0973b81658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:47.529031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:47.574031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad00415-28e7-4db7-a984-7a0973b81658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:47.575467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad00415-28e7-4db7-a984-7a0973b81658 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:47.646513 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28873d-c82e-4f6c-a48f-0faa834e7856 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:47.674275 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 547/1095] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 14:39:47.679943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad00415-28e7-4db7-a984-7a0973b81658 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:47.730523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:47.745640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28873d-c82e-4f6c-a48f-0faa834e7856 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:47.778350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad00415-28e7-4db7-a984-7a0973b81658 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:47.778848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbff0f3-e074-4118-9d9a-a486ef61699a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:47.779467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbff0f3-e074-4118-9d9a-a486ef61699a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:47.779773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbff0f3-e074-4118-9d9a-a486ef61699a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:47.780618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbff0f3-e074-4118-9d9a-a486ef61699a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:47.783394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:47.791608 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbff0f3-e074-4118-9d9a-a486ef61699a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-8bbff0f3-e074-4118-9d9a-a486ef61699a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:39:47.796341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:47.797414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:39:47.800670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 549/1096] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:39:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:47.803270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbff0f3-e074-4118-9d9a-a486ef61699a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:47.812643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38400708-75e9-4b3d-aee8-3d239750e4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:47.813303 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400708-75e9-4b3d-aee8-3d239750e4ea None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:47.813590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400708-75e9-4b3d-aee8-3d239750e4ea None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:47.813861 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400708-75e9-4b3d-aee8-3d239750e4ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:47.824397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38400708-75e9-4b3d-aee8-3d239750e4ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:47.825838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38400708-75e9-4b3d-aee8-3d239750e4ea None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-38400708-75e9-4b3d-aee8-3d239750e4ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:39:47.840238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:47.841164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:39:47.842059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:39:47.842059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:39:47.842059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:47.842059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 548/1097] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:47.847426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38400708-75e9-4b3d-aee8-3d239750e4ea None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:47.902057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbff0f3-e074-4118-9d9a-a486ef61699a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:47.935408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6ff8a-d389-41f3-93e6-b64a61df0cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:47.936089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6ff8a-d389-41f3-93e6-b64a61df0cb8 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:47.936365 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6ff8a-d389-41f3-93e6-b64a61df0cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:47.936635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6ff8a-d389-41f3-93e6-b64a61df0cb8 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:47.937295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6ff8a-d389-41f3-93e6-b64a61df0cb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:47.937598 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6ff8a-d389-41f3-93e6-b64a61df0cb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:47.938967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6ff8a-d389-41f3-93e6-b64a61df0cb8 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:47.946681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 550/1098] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:39:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:47.947439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38400708-75e9-4b3d-aee8-3d239750e4ea None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:47.949792 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:47.952834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6ff8a-d389-41f3-93e6-b64a61df0cb8 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:47.983537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21340229-7d68-4ff2-b7c0-70efe2c3d140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:47.984100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21340229-7d68-4ff2-b7c0-70efe2c3d140 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:47.984379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21340229-7d68-4ff2-b7c0-70efe2c3d140 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:47.984650 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21340229-7d68-4ff2-b7c0-70efe2c3d140 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:47.985284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21340229-7d68-4ff2-b7c0-70efe2c3d140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:47.985587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21340229-7d68-4ff2-b7c0-70efe2c3d140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:47.986956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21340229-7d68-4ff2-b7c0-70efe2c3d140 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:47.998311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 549/1099] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:48.004105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21340229-7d68-4ff2-b7c0-70efe2c3d140 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:48.008960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:48.050791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6ff8a-d389-41f3-93e6-b64a61df0cb8 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:48.083405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd50ed9-f02d-42d8-8f8c-47e0d0bc2484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:48.084058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd50ed9-f02d-42d8-8f8c-47e0d0bc2484 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:48.084335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd50ed9-f02d-42d8-8f8c-47e0d0bc2484 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:48.084604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd50ed9-f02d-42d8-8f8c-47e0d0bc2484 None admin] PATH_INFO: `/v3/users/8495719c77c84988b1255e7670dee0c0` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:48.085719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd50ed9-f02d-42d8-8f8c-47e0d0bc2484 None admin] RBAC: Authorizing `identity:get_user(user_id=8495719c77c84988b1255e7670dee0c0)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:48.087141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd50ed9-f02d-42d8-8f8c-47e0d0bc2484 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:48.088755 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 551/1100] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:39:47 2022] GET /identity/v3/users/8495719c77c84988b1255e7670dee0c0 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:48.095042 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd50ed9-f02d-42d8-8f8c-47e0d0bc2484 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:48.101709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21340229-7d68-4ff2-b7c0-70efe2c3d140 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:48.135780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cc424-e2ff-4b31-986c-f2550aea5389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:48.136406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cc424-e2ff-4b31-986c-f2550aea5389 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:48.136689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cc424-e2ff-4b31-986c-f2550aea5389 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:48.136964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cc424-e2ff-4b31-986c-f2550aea5389 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:48.144562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cc424-e2ff-4b31-986c-f2550aea5389 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:48.146005 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cc424-e2ff-4b31-986c-f2550aea5389 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-d75cc424-e2ff-4b31-986c-f2550aea5389 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:39:48.157438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:39:48.158405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:39:48.159305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:39:48.159305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:39:48.159305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:39:48.159305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:39:48.159305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:39:48.159305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:39:48.159305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:39:48.159305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:39:48.159305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 550/1101] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:39:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:39:48.164327 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cc424-e2ff-4b31-986c-f2550aea5389 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:48.189218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:48.193124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd50ed9-f02d-42d8-8f8c-47e0d0bc2484 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:48.217617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:48.226436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da32175-6823-4782-bb08-31e07929781a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:48.227032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da32175-6823-4782-bb08-31e07929781a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:48.227304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da32175-6823-4782-bb08-31e07929781a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:48.227572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da32175-6823-4782-bb08-31e07929781a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:48.262902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cc424-e2ff-4b31-986c-f2550aea5389 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:48.291922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da32175-6823-4782-bb08-31e07929781a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:48.293366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da32175-6823-4782-bb08-31e07929781a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:48.295442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7fd1a-e41b-433a-a376-597adf4f85e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:48.296075 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7fd1a-e41b-433a-a376-597adf4f85e4 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:48.296352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7fd1a-e41b-433a-a376-597adf4f85e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:48.296617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7fd1a-e41b-433a-a376-597adf4f85e4 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:48.297460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7fd1a-e41b-433a-a376-597adf4f85e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:48.297820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7fd1a-e41b-433a-a376-597adf4f85e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:48.299320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7fd1a-e41b-433a-a376-597adf4f85e4 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:48.311966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 551/1102] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:39:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:48.318090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7fd1a-e41b-433a-a376-597adf4f85e4 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:48.388885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 552/1103] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 14:39:48.395273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da32175-6823-4782-bb08-31e07929781a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:48.400344 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:48.417378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7fd1a-e41b-433a-a376-597adf4f85e4 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:48.421867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:48.451284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b79d1-15ff-45fb-a605-5f6a5a45d163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:48.452477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b79d1-15ff-45fb-a605-5f6a5a45d163 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:48.452779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b79d1-15ff-45fb-a605-5f6a5a45d163 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:48.453048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b79d1-15ff-45fb-a605-5f6a5a45d163 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:48.453764 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b79d1-15ff-45fb-a605-5f6a5a45d163 None admin] RBAC: Adding query filter params (role.id=b5a4a4e7850147889e8dfd1344c75440, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=3f3efa9f3eb24864943b68d241435817) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:48.454081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b79d1-15ff-45fb-a605-5f6a5a45d163 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:48.455500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b79d1-15ff-45fb-a605-5f6a5a45d163 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:48.461095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 552/1104] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:39:48 2022] GET /identity/v3/role_assignments?user.id=3f3efa9f3eb24864943b68d241435817&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=b5a4a4e7850147889e8dfd1344c75440 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:48.504026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da32175-6823-4782-bb08-31e07929781a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:48.539521 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b2af2-7aac-424e-ba1c-78de3b5827e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:48.540118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b2af2-7aac-424e-ba1c-78de3b5827e3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:48.540395 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b2af2-7aac-424e-ba1c-78de3b5827e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:48.540665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b2af2-7aac-424e-ba1c-78de3b5827e3 None admin] PATH_INFO: `/v3/projects/b2c2806854644a1eabfa5eefef46881b` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:48.541829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b2af2-7aac-424e-ba1c-78de3b5827e3 None admin] RBAC: Authorizing `identity:get_project(project_id=b2c2806854644a1eabfa5eefef46881b)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:48.543266 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b2af2-7aac-424e-ba1c-78de3b5827e3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:48.550456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 553/1105] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:39:48 2022] GET /identity/v3/projects/b2c2806854644a1eabfa5eefef46881b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:39:48.551668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b79d1-15ff-45fb-a605-5f6a5a45d163 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:48.652891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b79d1-15ff-45fb-a605-5f6a5a45d163 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:48.687591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8405b-c484-4821-ae5d-aefbaca91010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:48.688227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8405b-c484-4821-ae5d-aefbaca91010 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:48.688503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8405b-c484-4821-ae5d-aefbaca91010 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:48.688780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8405b-c484-4821-ae5d-aefbaca91010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:48.689490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8405b-c484-4821-ae5d-aefbaca91010 None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=b2c2806854644a1eabfa5eefef46881b, user.id=8495719c77c84988b1255e7670dee0c0) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:48.689790 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8405b-c484-4821-ae5d-aefbaca91010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:48.691224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8405b-c484-4821-ae5d-aefbaca91010 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:48.697013 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 553/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:39:48 2022] GET /identity/v3/role_assignments?user.id=8495719c77c84988b1255e7670dee0c0&scope.project.id=b2c2806854644a1eabfa5eefef46881b&role.id=9a439cf8eef34ada85428b378878e63e => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:50.606766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1d180-9408-44ba-bc7e-626b264eccb2 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:50.607295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1d180-9408-44ba-bc7e-626b264eccb2 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:50.607295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1d180-9408-44ba-bc7e-626b264eccb2 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:50.609252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 554/1107] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:50.614310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3fe75-8a1e-4c94-8bb1-57528ce0f96b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:50.614716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3fe75-8a1e-4c94-8bb1-57528ce0f96b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:50.614851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3fe75-8a1e-4c94-8bb1-57528ce0f96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:50.633648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8ed3fe75-8a1e-4c94-8bb1-57528ce0f96b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:50.634820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8ed3fe75-8a1e-4c94-8bb1-57528ce0f96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:50.641899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:50.698306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 554/1108] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:50.963820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1d180-9408-44ba-bc7e-626b264eccb2 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:51.054252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:51.061758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1d180-9408-44ba-bc7e-626b264eccb2 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:51.094972 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc93fee-b822-4181-babe-f28dcaef7261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:51.095576 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc93fee-b822-4181-babe-f28dcaef7261 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:51.095878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc93fee-b822-4181-babe-f28dcaef7261 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:51.096149 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc93fee-b822-4181-babe-f28dcaef7261 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:51.097234 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc93fee-b822-4181-babe-f28dcaef7261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:51.098645 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc93fee-b822-4181-babe-f28dcaef7261 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:51.100190 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 555/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:39:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:51.105901 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3fe75-8a1e-4c94-8bb1-57528ce0f96b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:51.203047 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3fe75-8a1e-4c94-8bb1-57528ce0f96b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:51.235180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d55a5-9d3e-48c5-b272-3f9b1a104f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:51.235860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d55a5-9d3e-48c5-b272-3f9b1a104f63 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:51.236182 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d55a5-9d3e-48c5-b272-3f9b1a104f63 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:51.236466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d55a5-9d3e-48c5-b272-3f9b1a104f63 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:51.237169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d55a5-9d3e-48c5-b272-3f9b1a104f63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:51.238659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d55a5-9d3e-48c5-b272-3f9b1a104f63 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:51.254804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:51.261481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 555/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:39:51 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:39:53.156363 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80558ea5-1e1d-45bf-94d7-761dc9da67c8 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:53.156870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80558ea5-1e1d-45bf-94d7-761dc9da67c8 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:53.156870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80558ea5-1e1d-45bf-94d7-761dc9da67c8 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:53.158117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 556/1111] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:53.163290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e8fe0-f789-4e29-9376-a550eeef8320 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:53.163714 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e8fe0-f789-4e29-9376-a550eeef8320 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:53.163868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e8fe0-f789-4e29-9376-a550eeef8320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:53.181361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-2f3e8fe0-f789-4e29-9376-a550eeef8320 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:53.182546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2f3e8fe0-f789-4e29-9376-a550eeef8320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:53.246530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 556/1112] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:53.521436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80558ea5-1e1d-45bf-94d7-761dc9da67c8 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:53.617719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80558ea5-1e1d-45bf-94d7-761dc9da67c8 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:53.651737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc4d3f-af0d-4b51-abe5-fae570cb2db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:53.652353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc4d3f-af0d-4b51-abe5-fae570cb2db5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:53.652627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc4d3f-af0d-4b51-abe5-fae570cb2db5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:53.652894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc4d3f-af0d-4b51-abe5-fae570cb2db5 None admin] PATH_INFO: `/v3/roles/9df5a7d7691c49b9a78f0db0a050b7fe` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:53.654245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc4d3f-af0d-4b51-abe5-fae570cb2db5 None admin] RBAC: Authorizing `identity:get_role(role_id=9df5a7d7691c49b9a78f0db0a050b7fe)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:53.655620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc4d3f-af0d-4b51-abe5-fae570cb2db5 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:53.656938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 557/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:39:53 2022] GET /identity/v3/roles/9df5a7d7691c49b9a78f0db0a050b7fe => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:53.662721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e8fe0-f789-4e29-9376-a550eeef8320 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:53.758414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e8fe0-f789-4e29-9376-a550eeef8320 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:53.784108 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:53.791901 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046f89d-d2b5-41ba-9bfb-2e897a0ee43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:53.792575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046f89d-d2b5-41ba-9bfb-2e897a0ee43d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:53.792800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046f89d-d2b5-41ba-9bfb-2e897a0ee43d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:53.793071 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046f89d-d2b5-41ba-9bfb-2e897a0ee43d None admin] PATH_INFO: `/v3/users/8800ad25b6aa478ab66393014b56aee3` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:53.806273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046f89d-d2b5-41ba-9bfb-2e897a0ee43d None admin] RBAC: Authorizing `identity:get_user(user_id=8800ad25b6aa478ab66393014b56aee3)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:53.807721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046f89d-d2b5-41ba-9bfb-2e897a0ee43d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:53.809302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 557/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:39:53 2022] GET /identity/v3/users/8800ad25b6aa478ab66393014b56aee3 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:53.815777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc4d3f-af0d-4b51-abe5-fae570cb2db5 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:53.845856 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:53.913444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc4d3f-af0d-4b51-abe5-fae570cb2db5 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:53.946788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949ae02-1339-4901-aeeb-12d66be3c9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:53.947384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949ae02-1339-4901-aeeb-12d66be3c9df None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:53.947663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949ae02-1339-4901-aeeb-12d66be3c9df None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:53.947965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949ae02-1339-4901-aeeb-12d66be3c9df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:54.014420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949ae02-1339-4901-aeeb-12d66be3c9df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:54.015886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949ae02-1339-4901-aeeb-12d66be3c9df None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:54.044022 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:54.112852 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 558/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:39:54.118801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046f89d-d2b5-41ba-9bfb-2e897a0ee43d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:54.212688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046f89d-d2b5-41ba-9bfb-2e897a0ee43d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:54.245348 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd26caa-0c51-43bc-8473-ac63741074d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:54.245950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd26caa-0c51-43bc-8473-ac63741074d5 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:54.246223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd26caa-0c51-43bc-8473-ac63741074d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:54.246490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd26caa-0c51-43bc-8473-ac63741074d5 None admin] PATH_INFO: `/v3/projects/b2c2806854644a1eabfa5eefef46881b` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:54.247579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd26caa-0c51-43bc-8473-ac63741074d5 None admin] RBAC: Authorizing `identity:get_project(project_id=b2c2806854644a1eabfa5eefef46881b)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:54.249092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd26caa-0c51-43bc-8473-ac63741074d5 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:54.250933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 558/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:39:54 2022] GET /identity/v3/projects/b2c2806854644a1eabfa5eefef46881b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:54.256968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949ae02-1339-4901-aeeb-12d66be3c9df None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:54.352005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949ae02-1339-4901-aeeb-12d66be3c9df None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:54.379359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:54.385438 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac42be2-05f4-4290-b48c-c767b1eff6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:54.386024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac42be2-05f4-4290-b48c-c767b1eff6b5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:54.386300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac42be2-05f4-4290-b48c-c767b1eff6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:54.386569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac42be2-05f4-4290-b48c-c767b1eff6b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:54.387282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac42be2-05f4-4290-b48c-c767b1eff6b5 None admin] RBAC: Adding query filter params (role.id=9df5a7d7691c49b9a78f0db0a050b7fe, scope.project.id=b2c2806854644a1eabfa5eefef46881b, user.id=8800ad25b6aa478ab66393014b56aee3) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:39:54.387585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac42be2-05f4-4290-b48c-c767b1eff6b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:54.389028 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac42be2-05f4-4290-b48c-c767b1eff6b5 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:54.394176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 559/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:39:54 2022] GET /identity/v3/role_assignments?user.id=8800ad25b6aa478ab66393014b56aee3&scope.project.id=b2c2806854644a1eabfa5eefef46881b&role.id=9df5a7d7691c49b9a78f0db0a050b7fe => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:56.256758 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57112a-7711-4d9f-86e6-2d5c7fc757ba None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:56.257300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57112a-7711-4d9f-86e6-2d5c7fc757ba None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:56.257300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57112a-7711-4d9f-86e6-2d5c7fc757ba None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:56.258641 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 559/1118] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:56.263929 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1bf95-3107-461e-af06-3b7f561ae37b None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:56.264854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1bf95-3107-461e-af06-3b7f561ae37b None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:56.265002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1bf95-3107-461e-af06-3b7f561ae37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:56.282925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-20d1bf95-3107-461e-af06-3b7f561ae37b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:56.284140 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-20d1bf95-3107-461e-af06-3b7f561ae37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:56.347100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 560/1119] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 14:39:56.616483 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57112a-7711-4d9f-86e6-2d5c7fc757ba None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:56.672044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:56.716229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57112a-7711-4d9f-86e6-2d5c7fc757ba None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:56.749464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9d357-54e1-4224-82a6-dbdf8d7cba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:56.750340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9d357-54e1-4224-82a6-dbdf8d7cba11 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:56.750618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9d357-54e1-4224-82a6-dbdf8d7cba11 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:56.750899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9d357-54e1-4224-82a6-dbdf8d7cba11 None admin] PATH_INFO: `/v3/roles/9df5a7d7691c49b9a78f0db0a050b7fe` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:56.752305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9d357-54e1-4224-82a6-dbdf8d7cba11 None admin] RBAC: Authorizing `identity:get_role(role_id=9df5a7d7691c49b9a78f0db0a050b7fe)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:56.753672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9d357-54e1-4224-82a6-dbdf8d7cba11 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:56.754959 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 560/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:39:56 2022] GET /identity/v3/roles/9df5a7d7691c49b9a78f0db0a050b7fe => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:56.760843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1bf95-3107-461e-af06-3b7f561ae37b None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:56.854446 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1bf95-3107-461e-af06-3b7f561ae37b None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:56.876337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:56.886774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffd3a8-70b2-4b1d-a17c-86554732c70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:56.887383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffd3a8-70b2-4b1d-a17c-86554732c70e None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:56.887656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffd3a8-70b2-4b1d-a17c-86554732c70e None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:56.887955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffd3a8-70b2-4b1d-a17c-86554732c70e None admin] PATH_INFO: `/v3/users/8800ad25b6aa478ab66393014b56aee3` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:56.889582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffd3a8-70b2-4b1d-a17c-86554732c70e None admin] RBAC: Authorizing `identity:get_user(user_id=8800ad25b6aa478ab66393014b56aee3)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:56.890991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffd3a8-70b2-4b1d-a17c-86554732c70e None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:56.892562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 561/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:39:56 2022] GET /identity/v3/users/8800ad25b6aa478ab66393014b56aee3 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:56.898204 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9d357-54e1-4224-82a6-dbdf8d7cba11 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:56.993586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9d357-54e1-4224-82a6-dbdf8d7cba11 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:57.015558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:57.026899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54771c1-20c1-4302-abaa-02c083bd5b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:57.027512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54771c1-20c1-4302-abaa-02c083bd5b75 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:57.027813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54771c1-20c1-4302-abaa-02c083bd5b75 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:57.028082 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54771c1-20c1-4302-abaa-02c083bd5b75 None admin] PATH_INFO: `/v3/projects/b2c2806854644a1eabfa5eefef46881b` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:57.029179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54771c1-20c1-4302-abaa-02c083bd5b75 None admin] RBAC: Authorizing `identity:get_project(project_id=b2c2806854644a1eabfa5eefef46881b)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:57.031201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54771c1-20c1-4302-abaa-02c083bd5b75 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:57.032839 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 561/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:39:56 2022] GET /identity/v3/projects/b2c2806854644a1eabfa5eefef46881b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:57.038747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffd3a8-70b2-4b1d-a17c-86554732c70e None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:57.132588 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffd3a8-70b2-4b1d-a17c-86554732c70e None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:57.163884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d1bdd-414b-4d3a-a325-93f6a7ac14cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:57.164510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d1bdd-414b-4d3a-a325-93f6a7ac14cf None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:57.164783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d1bdd-414b-4d3a-a325-93f6a7ac14cf None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:57.165051 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d1bdd-414b-4d3a-a325-93f6a7ac14cf None admin] PATH_INFO: `/v3/projects/b2c2806854644a1eabfa5eefef46881b/users/8800ad25b6aa478ab66393014b56aee3/roles/9df5a7d7691c49b9a78f0db0a050b7fe` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:57.167131 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d1bdd-414b-4d3a-a325-93f6a7ac14cf None admin] RBAC: Authorizing `identity:create_grant(project_id=b2c2806854644a1eabfa5eefef46881b, user_id=8800ad25b6aa478ab66393014b56aee3, role_id=9df5a7d7691c49b9a78f0db0a050b7fe)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:57.168573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d1bdd-414b-4d3a-a325-93f6a7ac14cf None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:57.181151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 562/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:39:57 2022] PUT /identity/v3/projects/b2c2806854644a1eabfa5eefef46881b/users/8800ad25b6aa478ab66393014b56aee3/roles/9df5a7d7691c49b9a78f0db0a050b7fe => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:39:59.020520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f80b52-5221-4a10-9eea-de9b6736048a None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:59.021053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f80b52-5221-4a10-9eea-de9b6736048a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:59.021053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f80b52-5221-4a10-9eea-de9b6736048a None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:59.022371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 562/1124] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:39:59.027629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ecc5a-ea3f-4b6f-b8cb-985449c00486 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:59.028069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ecc5a-ea3f-4b6f-b8cb-985449c00486 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:59.028185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ecc5a-ea3f-4b6f-b8cb-985449c00486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:59.046250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-192ecc5a-ea3f-4b6f-b8cb-985449c00486 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:39:59.047427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-192ecc5a-ea3f-4b6f-b8cb-985449c00486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:39:59.091535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:59.111434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 563/1125] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:39:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:39:59.385949 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f80b52-5221-4a10-9eea-de9b6736048a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:59.483095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f80b52-5221-4a10-9eea-de9b6736048a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:59.515389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a3e88-0558-47be-b726-2d745ae66427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:59.516019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a3e88-0558-47be-b726-2d745ae66427 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:59.516291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a3e88-0558-47be-b726-2d745ae66427 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:59.516556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a3e88-0558-47be-b726-2d745ae66427 None admin] PATH_INFO: `/v3/roles/9df5a7d7691c49b9a78f0db0a050b7fe` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:59.517904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a3e88-0558-47be-b726-2d745ae66427 None admin] RBAC: Authorizing `identity:get_role(role_id=9df5a7d7691c49b9a78f0db0a050b7fe)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:59.519251 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a3e88-0558-47be-b726-2d745ae66427 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:59.520553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 563/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:39:59 2022] GET /identity/v3/roles/9df5a7d7691c49b9a78f0db0a050b7fe => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:59.526258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ecc5a-ea3f-4b6f-b8cb-985449c00486 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:59.621331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ecc5a-ea3f-4b6f-b8cb-985449c00486 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:59.653329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44029b8-e69b-4c79-94fb-1883f1ba5af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:59.653891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44029b8-e69b-4c79-94fb-1883f1ba5af1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:59.654168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44029b8-e69b-4c79-94fb-1883f1ba5af1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:59.654435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44029b8-e69b-4c79-94fb-1883f1ba5af1 None admin] PATH_INFO: `/v3/users/8800ad25b6aa478ab66393014b56aee3` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:59.656077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44029b8-e69b-4c79-94fb-1883f1ba5af1 None admin] RBAC: Authorizing `identity:get_user(user_id=8800ad25b6aa478ab66393014b56aee3)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:59.657501 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44029b8-e69b-4c79-94fb-1883f1ba5af1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:59.659059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 564/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:39:59 2022] GET /identity/v3/users/8800ad25b6aa478ab66393014b56aee3 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:39:59.670104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a3e88-0558-47be-b726-2d745ae66427 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:39:59.727109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:59.767185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a3e88-0558-47be-b726-2d745ae66427 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:39:59.799181 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acc235-4778-4420-a560-4ce77c79f5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:39:59.799790 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acc235-4778-4420-a560-4ce77c79f5bf None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:39:59.800065 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acc235-4778-4420-a560-4ce77c79f5bf None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:39:59.800331 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acc235-4778-4420-a560-4ce77c79f5bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:39:59.864629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acc235-4778-4420-a560-4ce77c79f5bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:39:59.866052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acc235-4778-4420-a560-4ce77c79f5bf None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:39:59.920064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:39:59.961217 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 564/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:39:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:39:59.967060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44029b8-e69b-4c79-94fb-1883f1ba5af1 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:00.014670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:00.061146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44029b8-e69b-4c79-94fb-1883f1ba5af1 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:00.092375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee97ae9-df94-4b40-95eb-06649db32dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:00.092979 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee97ae9-df94-4b40-95eb-06649db32dc3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:00.093253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee97ae9-df94-4b40-95eb-06649db32dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:00.093519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee97ae9-df94-4b40-95eb-06649db32dc3 None admin] PATH_INFO: `/v3/projects/b2c2806854644a1eabfa5eefef46881b` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:00.094610 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee97ae9-df94-4b40-95eb-06649db32dc3 None admin] RBAC: Authorizing `identity:get_project(project_id=b2c2806854644a1eabfa5eefef46881b)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:00.096047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee97ae9-df94-4b40-95eb-06649db32dc3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:00.097631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 565/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:39:59 2022] GET /identity/v3/projects/b2c2806854644a1eabfa5eefef46881b => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:00.103559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acc235-4778-4420-a560-4ce77c79f5bf None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:00.197631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acc235-4778-4420-a560-4ce77c79f5bf None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:00.230220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41472391-9831-4bf2-905b-df9d926216a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:00.230798 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41472391-9831-4bf2-905b-df9d926216a1 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:00.231072 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41472391-9831-4bf2-905b-df9d926216a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:00.231340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41472391-9831-4bf2-905b-df9d926216a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:00.232087 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41472391-9831-4bf2-905b-df9d926216a1 None admin] RBAC: Adding query filter params (role.id=9df5a7d7691c49b9a78f0db0a050b7fe, scope.project.id=b2c2806854644a1eabfa5eefef46881b, user.id=8800ad25b6aa478ab66393014b56aee3) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:40:00.232391 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41472391-9831-4bf2-905b-df9d926216a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:00.233821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41472391-9831-4bf2-905b-df9d926216a1 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:00.239322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 565/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:40:00 2022] GET /identity/v3/role_assignments?user.id=8800ad25b6aa478ab66393014b56aee3&scope.project.id=b2c2806854644a1eabfa5eefef46881b&role.id=9df5a7d7691c49b9a78f0db0a050b7fe => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:02.133696 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc63408-7601-4558-8714-7d02433e7693 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:02.134273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc63408-7601-4558-8714-7d02433e7693 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:02.134273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc63408-7601-4558-8714-7d02433e7693 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:02.135560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 566/1131] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:40:02.141320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccb84f-30fc-4d9a-919a-0902ce16c016 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:02.141748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccb84f-30fc-4d9a-919a-0902ce16c016 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:02.141875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccb84f-30fc-4d9a-919a-0902ce16c016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:02.160828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4cccb84f-30fc-4d9a-919a-0902ce16c016 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:40:02.161962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4cccb84f-30fc-4d9a-919a-0902ce16c016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:40:02.177843 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:02.226671 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 566/1132] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:40:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:40:02.494860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc63408-7601-4558-8714-7d02433e7693 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:02.595195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc63408-7601-4558-8714-7d02433e7693 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:02.612757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:02.629115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04cca3-870a-4db8-a75a-c77baaf2de9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:02.629735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04cca3-870a-4db8-a75a-c77baaf2de9a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:02.630008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04cca3-870a-4db8-a75a-c77baaf2de9a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:02.630274 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04cca3-870a-4db8-a75a-c77baaf2de9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:02.631372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04cca3-870a-4db8-a75a-c77baaf2de9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:02.632826 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04cca3-870a-4db8-a75a-c77baaf2de9a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:02.634359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 567/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:40:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:02.640200 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccb84f-30fc-4d9a-919a-0902ce16c016 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:02.734158 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccb84f-30fc-4d9a-919a-0902ce16c016 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:02.766367 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554efb7-66f6-4470-aeac-51736b4a9a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:02.766892 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554efb7-66f6-4470-aeac-51736b4a9a55 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:02.767164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554efb7-66f6-4470-aeac-51736b4a9a55 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:02.767431 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554efb7-66f6-4470-aeac-51736b4a9a55 None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:02.768093 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554efb7-66f6-4470-aeac-51736b4a9a55 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:02.769501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554efb7-66f6-4470-aeac-51736b4a9a55 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:02.795393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 567/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:40:02 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:40:04.692989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae1eee-fdb5-4f8b-bac1-fcfa1536863f None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:04.693531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae1eee-fdb5-4f8b-bac1-fcfa1536863f None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:04.693531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae1eee-fdb5-4f8b-bac1-fcfa1536863f None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:04.694839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 568/1135] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:40:04.699984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b31bf-97ca-4d6d-b1c6-acbcc994b6ba None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:04.700379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b31bf-97ca-4d6d-b1c6-acbcc994b6ba None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:04.700530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b31bf-97ca-4d6d-b1c6-acbcc994b6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:04.707594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:04.719205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-d93b31bf-97ca-4d6d-b1c6-acbcc994b6ba None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:40:04.720414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d93b31bf-97ca-4d6d-b1c6-acbcc994b6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:40:04.783663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 568/1136] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:40:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:40:05.054713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae1eee-fdb5-4f8b-bac1-fcfa1536863f None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:05.150470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae1eee-fdb5-4f8b-bac1-fcfa1536863f None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:05.181966 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e9cd8-022b-40cf-8eec-533aa88c47e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:05.182564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e9cd8-022b-40cf-8eec-533aa88c47e1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:05.182838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e9cd8-022b-40cf-8eec-533aa88c47e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:05.183105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e9cd8-022b-40cf-8eec-533aa88c47e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:05.184216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e9cd8-022b-40cf-8eec-533aa88c47e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:05.185622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e9cd8-022b-40cf-8eec-533aa88c47e1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:05.187210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 569/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:40:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:05.192801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b31bf-97ca-4d6d-b1c6-acbcc994b6ba None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:05.287535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b31bf-97ca-4d6d-b1c6-acbcc994b6ba None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:05.319857 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fce71-f47a-4696-815f-5fa39bdee18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:05.320391 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fce71-f47a-4696-815f-5fa39bdee18b None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:05.320667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fce71-f47a-4696-815f-5fa39bdee18b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:05.320933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fce71-f47a-4696-815f-5fa39bdee18b None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:05.321575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fce71-f47a-4696-815f-5fa39bdee18b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:05.322989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fce71-f47a-4696-815f-5fa39bdee18b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:05.331399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:05.344697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 569/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:40:05 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:40:07.249736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99645fd2-7fd4-4dc6-befb-49235fa79e52 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:07.250531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99645fd2-7fd4-4dc6-befb-49235fa79e52 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:07.250531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99645fd2-7fd4-4dc6-befb-49235fa79e52 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:07.251914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 570/1139] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:40:07.256923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18588f-25a2-4dfc-b054-1b23a1579ebb None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:07.257352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18588f-25a2-4dfc-b054-1b23a1579ebb None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:07.257447 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18588f-25a2-4dfc-b054-1b23a1579ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:07.275372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-ed18588f-25a2-4dfc-b054-1b23a1579ebb None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:40:07.276578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ed18588f-25a2-4dfc-b054-1b23a1579ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:40:07.343580 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 570/1140] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:40:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:40:07.606490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99645fd2-7fd4-4dc6-befb-49235fa79e52 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:07.660997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:07.705474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99645fd2-7fd4-4dc6-befb-49235fa79e52 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:07.737429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd047b-2957-49e0-ad18-750a57c3ba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:07.738039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd047b-2957-49e0-ad18-750a57c3ba25 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:07.738315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd047b-2957-49e0-ad18-750a57c3ba25 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:07.738584 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd047b-2957-49e0-ad18-750a57c3ba25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:07.748802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd047b-2957-49e0-ad18-750a57c3ba25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-b4cd047b-2957-49e0-ad18-750a57c3ba25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:40:07.753680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:40:07.754861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:40:07.755960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 571/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:40:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:40:07.760833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18588f-25a2-4dfc-b054-1b23a1579ebb None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:07.854523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18588f-25a2-4dfc-b054-1b23a1579ebb None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:07.873888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:07.887909 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d92283-6ab2-4ec0-b5fb-52e7473fe168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:07.888501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d92283-6ab2-4ec0-b5fb-52e7473fe168 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:07.888778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d92283-6ab2-4ec0-b5fb-52e7473fe168 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:07.889584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d92283-6ab2-4ec0-b5fb-52e7473fe168 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:07.890235 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d92283-6ab2-4ec0-b5fb-52e7473fe168 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:40:07.890536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d92283-6ab2-4ec0-b5fb-52e7473fe168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:07.891922 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d92283-6ab2-4ec0-b5fb-52e7473fe168 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:07.902043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 571/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:40:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:07.907799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd047b-2957-49e0-ad18-750a57c3ba25 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:08.004085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd047b-2957-49e0-ad18-750a57c3ba25 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:08.017170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:08.036139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce73f4-4ba9-4d76-851f-655287087bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:08.036749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce73f4-4ba9-4d76-851f-655287087bb1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:08.037023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce73f4-4ba9-4d76-851f-655287087bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:08.037291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce73f4-4ba9-4d76-851f-655287087bb1 None admin] PATH_INFO: `/v3/users/c8c74b1548fc4d6f969ffaf3c75c18bc` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:08.050054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce73f4-4ba9-4d76-851f-655287087bb1 None admin] RBAC: Authorizing `identity:get_user(user_id=c8c74b1548fc4d6f969ffaf3c75c18bc)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:08.051473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce73f4-4ba9-4d76-851f-655287087bb1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:08.053059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 572/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:40:07 2022] GET /identity/v3/users/c8c74b1548fc4d6f969ffaf3c75c18bc => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:08.058929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d92283-6ab2-4ec0-b5fb-52e7473fe168 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:08.155511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d92283-6ab2-4ec0-b5fb-52e7473fe168 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:08.186828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632925f0-ec74-48bb-935c-a0ec10fd417d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:08.187437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632925f0-ec74-48bb-935c-a0ec10fd417d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:08.187728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632925f0-ec74-48bb-935c-a0ec10fd417d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:08.187993 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632925f0-ec74-48bb-935c-a0ec10fd417d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:08.234012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:08.251863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632925f0-ec74-48bb-935c-a0ec10fd417d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:08.253311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632925f0-ec74-48bb-935c-a0ec10fd417d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:08.346060 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 572/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:40:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:40:08.351877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce73f4-4ba9-4d76-851f-655287087bb1 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:08.445035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce73f4-4ba9-4d76-851f-655287087bb1 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:08.477284 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45da865-a6d8-4dcf-a4b7-221a2d4decda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:08.477887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45da865-a6d8-4dcf-a4b7-221a2d4decda None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:08.478165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45da865-a6d8-4dcf-a4b7-221a2d4decda None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:08.478431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45da865-a6d8-4dcf-a4b7-221a2d4decda None admin] PATH_INFO: `/v3/projects/96d7e858fce54734989a93943d50ceca` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:08.479522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45da865-a6d8-4dcf-a4b7-221a2d4decda None admin] RBAC: Authorizing `identity:get_project(project_id=96d7e858fce54734989a93943d50ceca)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:08.480957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45da865-a6d8-4dcf-a4b7-221a2d4decda None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:08.482527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 573/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:40:08 2022] GET /identity/v3/projects/96d7e858fce54734989a93943d50ceca => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:40:08.488741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632925f0-ec74-48bb-935c-a0ec10fd417d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:08.566458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:08.584153 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632925f0-ec74-48bb-935c-a0ec10fd417d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:08.615598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aac6e7-c8e3-4d64-8d0a-b1335a327f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:08.616201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aac6e7-c8e3-4d64-8d0a-b1335a327f4e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:08.616473 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aac6e7-c8e3-4d64-8d0a-b1335a327f4e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:08.616736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aac6e7-c8e3-4d64-8d0a-b1335a327f4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:08.617440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aac6e7-c8e3-4d64-8d0a-b1335a327f4e None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=96d7e858fce54734989a93943d50ceca, user.id=c8c74b1548fc4d6f969ffaf3c75c18bc) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:40:08.617740 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aac6e7-c8e3-4d64-8d0a-b1335a327f4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:08.619136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aac6e7-c8e3-4d64-8d0a-b1335a327f4e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:08.624791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 573/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:40:08 2022] GET /identity/v3/role_assignments?user.id=c8c74b1548fc4d6f969ffaf3c75c18bc&scope.project.id=96d7e858fce54734989a93943d50ceca&role.id=9a439cf8eef34ada85428b378878e63e => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:10.481702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f460564-9742-43e6-94c1-e00a7ff42090 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:10.482245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f460564-9742-43e6-94c1-e00a7ff42090 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:10.482245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f460564-9742-43e6-94c1-e00a7ff42090 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:10.483568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 574/1147] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:40:10.488670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2dadb-7924-4e6c-ba37-224221532d04 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:10.489103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2dadb-7924-4e6c-ba37-224221532d04 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:10.489204 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2dadb-7924-4e6c-ba37-224221532d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:10.506876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-78f2dadb-7924-4e6c-ba37-224221532d04 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:40:10.508085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-78f2dadb-7924-4e6c-ba37-224221532d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:40:10.571230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 574/1148] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:40:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:40:10.839005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f460564-9742-43e6-94c1-e00a7ff42090 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:10.881412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:10.938811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f460564-9742-43e6-94c1-e00a7ff42090 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:10.971213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87c4d4-859e-4f87-8c0b-c276ddea9ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:10.971859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87c4d4-859e-4f87-8c0b-c276ddea9ade None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:10.972139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87c4d4-859e-4f87-8c0b-c276ddea9ade None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:10.972409 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87c4d4-859e-4f87-8c0b-c276ddea9ade None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:10.982027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87c4d4-859e-4f87-8c0b-c276ddea9ade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-ad87c4d4-859e-4f87-8c0b-c276ddea9ade None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:40:10.987171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:40:10.988308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:40:10.989515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 575/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:40:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:40:10.994234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2dadb-7924-4e6c-ba37-224221532d04 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:11.067288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:11.088933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2dadb-7924-4e6c-ba37-224221532d04 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:11.121291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245820b8-6e3b-40de-98e3-4bf197bdb654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:11.121828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245820b8-6e3b-40de-98e3-4bf197bdb654 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:11.122103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245820b8-6e3b-40de-98e3-4bf197bdb654 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:11.122371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245820b8-6e3b-40de-98e3-4bf197bdb654 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:11.123022 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245820b8-6e3b-40de-98e3-4bf197bdb654 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:40:11.123888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245820b8-6e3b-40de-98e3-4bf197bdb654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:11.125508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245820b8-6e3b-40de-98e3-4bf197bdb654 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:11.133196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 575/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:40:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:40:11.138878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87c4d4-859e-4f87-8c0b-c276ddea9ade None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:11.231219 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:11.234687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87c4d4-859e-4f87-8c0b-c276ddea9ade None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:11.267391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92d2c1-94b7-4989-a963-14439804943b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:11.268028 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92d2c1-94b7-4989-a963-14439804943b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:11.268303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92d2c1-94b7-4989-a963-14439804943b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:11.268571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92d2c1-94b7-4989-a963-14439804943b None admin] PATH_INFO: `/v3/users/c8c74b1548fc4d6f969ffaf3c75c18bc` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:11.269671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92d2c1-94b7-4989-a963-14439804943b None admin] RBAC: Authorizing `identity:get_user(user_id=c8c74b1548fc4d6f969ffaf3c75c18bc)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:11.271076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92d2c1-94b7-4989-a963-14439804943b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:11.272676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 576/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:40:11 2022] GET /identity/v3/users/c8c74b1548fc4d6f969ffaf3c75c18bc => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:11.278279 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245820b8-6e3b-40de-98e3-4bf197bdb654 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:11.375529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245820b8-6e3b-40de-98e3-4bf197bdb654 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:11.407735 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8550885-58fa-49d3-b4dc-4e854486c39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:11.408343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8550885-58fa-49d3-b4dc-4e854486c39a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:11.408618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8550885-58fa-49d3-b4dc-4e854486c39a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:11.408887 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8550885-58fa-49d3-b4dc-4e854486c39a None admin] PATH_INFO: `/v3/projects/96d7e858fce54734989a93943d50ceca` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:11.409982 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8550885-58fa-49d3-b4dc-4e854486c39a None admin] RBAC: Authorizing `identity:get_project(project_id=96d7e858fce54734989a93943d50ceca)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:11.411400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8550885-58fa-49d3-b4dc-4e854486c39a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:11.412997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 576/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:40:11 2022] GET /identity/v3/projects/96d7e858fce54734989a93943d50ceca => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:11.418645 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92d2c1-94b7-4989-a963-14439804943b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:11.513223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92d2c1-94b7-4989-a963-14439804943b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:11.544581 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2482de0-5013-478d-9c2f-461e1367eb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:11.545416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2482de0-5013-478d-9c2f-461e1367eb57 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:11.545689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2482de0-5013-478d-9c2f-461e1367eb57 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:11.545956 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2482de0-5013-478d-9c2f-461e1367eb57 None admin] PATH_INFO: `/v3/projects/96d7e858fce54734989a93943d50ceca/users/c8c74b1548fc4d6f969ffaf3c75c18bc/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:11.547830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2482de0-5013-478d-9c2f-461e1367eb57 None admin] RBAC: Authorizing `identity:create_grant(project_id=96d7e858fce54734989a93943d50ceca, user_id=c8c74b1548fc4d6f969ffaf3c75c18bc, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:11.549234 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2482de0-5013-478d-9c2f-461e1367eb57 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:11.562535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 577/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:40:11 2022] PUT /identity/v3/projects/96d7e858fce54734989a93943d50ceca/users/c8c74b1548fc4d6f969ffaf3c75c18bc/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:40:13.403401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be1c02-8def-447c-a75a-dc5ea41bf806 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:13.403986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be1c02-8def-447c-a75a-dc5ea41bf806 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:13.403986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be1c02-8def-447c-a75a-dc5ea41bf806 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:13.405840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 577/1154] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:40:13.410898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beff7ba-0f71-4aa3-bd97-266b4cb2c34a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:13.411337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beff7ba-0f71-4aa3-bd97-266b4cb2c34a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:13.411432 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beff7ba-0f71-4aa3-bd97-266b4cb2c34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:13.428970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-8beff7ba-0f71-4aa3-bd97-266b4cb2c34a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:40:13.430145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8beff7ba-0f71-4aa3-bd97-266b4cb2c34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:40:13.452770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:13.495692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 578/1155] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:40:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:40:13.758716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be1c02-8def-447c-a75a-dc5ea41bf806 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:13.767994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:13.855070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be1c02-8def-447c-a75a-dc5ea41bf806 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:13.886901 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf44d7-fa45-480b-b48f-64cebfae6150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:13.887502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf44d7-fa45-480b-b48f-64cebfae6150 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:13.887810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf44d7-fa45-480b-b48f-64cebfae6150 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:13.888075 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf44d7-fa45-480b-b48f-64cebfae6150 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:13.898315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf44d7-fa45-480b-b48f-64cebfae6150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-babf44d7-fa45-480b-b48f-64cebfae6150 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:40:13.902906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:40:13.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:40:13.905567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 578/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:40:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:40:13.910486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beff7ba-0f71-4aa3-bd97-266b4cb2c34a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:14.006412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beff7ba-0f71-4aa3-bd97-266b4cb2c34a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:14.039158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63417440-0eb7-4fe9-b3b3-24940ecff9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:14.039706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63417440-0eb7-4fe9-b3b3-24940ecff9d9 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:14.040001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63417440-0eb7-4fe9-b3b3-24940ecff9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:14.040274 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63417440-0eb7-4fe9-b3b3-24940ecff9d9 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:14.040932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63417440-0eb7-4fe9-b3b3-24940ecff9d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:40:14.041233 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63417440-0eb7-4fe9-b3b3-24940ecff9d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:14.042587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63417440-0eb7-4fe9-b3b3-24940ecff9d9 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:14.050207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 579/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:40:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:14.055954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf44d7-fa45-480b-b48f-64cebfae6150 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:14.118806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:14.153027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf44d7-fa45-480b-b48f-64cebfae6150 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:14.184711 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4a4b2-9141-4ca4-ae72-187997783abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:14.185319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4a4b2-9141-4ca4-ae72-187997783abd None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:14.185614 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4a4b2-9141-4ca4-ae72-187997783abd None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:14.185886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4a4b2-9141-4ca4-ae72-187997783abd None admin] PATH_INFO: `/v3/users/c8c74b1548fc4d6f969ffaf3c75c18bc` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:14.186975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4a4b2-9141-4ca4-ae72-187997783abd None admin] RBAC: Authorizing `identity:get_user(user_id=c8c74b1548fc4d6f969ffaf3c75c18bc)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:14.188418 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4a4b2-9141-4ca4-ae72-187997783abd None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:14.189949 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 579/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:40:14 2022] GET /identity/v3/users/c8c74b1548fc4d6f969ffaf3c75c18bc => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:14.195791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63417440-0eb7-4fe9-b3b3-24940ecff9d9 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:14.218117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:14.291709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63417440-0eb7-4fe9-b3b3-24940ecff9d9 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:14.322689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd4399-6b0b-4404-becf-238741ddc025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:14.323279 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd4399-6b0b-4404-becf-238741ddc025 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:14.323554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd4399-6b0b-4404-becf-238741ddc025 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:14.323853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd4399-6b0b-4404-becf-238741ddc025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:14.391039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd4399-6b0b-4404-becf-238741ddc025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:14.392504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd4399-6b0b-4404-becf-238741ddc025 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:14.411673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:14.486003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 580/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:40:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:40:14.491805 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4a4b2-9141-4ca4-ae72-187997783abd None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:14.587387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4a4b2-9141-4ca4-ae72-187997783abd None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:14.618450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b49e98-91ed-44ba-9771-97983d4e3dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:14.619048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b49e98-91ed-44ba-9771-97983d4e3dc3 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:14.619320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b49e98-91ed-44ba-9771-97983d4e3dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:14.619588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b49e98-91ed-44ba-9771-97983d4e3dc3 None admin] PATH_INFO: `/v3/projects/96d7e858fce54734989a93943d50ceca` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:14.620705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b49e98-91ed-44ba-9771-97983d4e3dc3 None admin] RBAC: Authorizing `identity:get_project(project_id=96d7e858fce54734989a93943d50ceca)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:14.622116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b49e98-91ed-44ba-9771-97983d4e3dc3 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:14.624228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 580/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:40:14 2022] GET /identity/v3/projects/96d7e858fce54734989a93943d50ceca => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:14.630464 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd4399-6b0b-4404-becf-238741ddc025 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:14.724512 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd4399-6b0b-4404-becf-238741ddc025 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:14.743892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:14.758471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18147927-ad64-4101-8734-f62382f61280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:14.759057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18147927-ad64-4101-8734-f62382f61280 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:14.759338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18147927-ad64-4101-8734-f62382f61280 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:14.759611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18147927-ad64-4101-8734-f62382f61280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:14.760355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18147927-ad64-4101-8734-f62382f61280 None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=96d7e858fce54734989a93943d50ceca, user.id=c8c74b1548fc4d6f969ffaf3c75c18bc) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:40:14.760663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18147927-ad64-4101-8734-f62382f61280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:14.762070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18147927-ad64-4101-8734-f62382f61280 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:14.767582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 581/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:40:14 2022] GET /identity/v3/role_assignments?user.id=c8c74b1548fc4d6f969ffaf3c75c18bc&scope.project.id=96d7e858fce54734989a93943d50ceca&role.id=9a439cf8eef34ada85428b378878e63e => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:16.652003 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bb8a5-547d-4cf4-ab80-77b790ff9927 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:16.652597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bb8a5-547d-4cf4-ab80-77b790ff9927 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:16.652597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bb8a5-547d-4cf4-ab80-77b790ff9927 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:16.653891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 581/1162] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:40:16.659730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0188c-69a6-435f-a828-28f14e8ee280 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:16.660131 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0188c-69a6-435f-a828-28f14e8ee280 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:16.660280 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0188c-69a6-435f-a828-28f14e8ee280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:16.678241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f1e0188c-69a6-435f-a828-28f14e8ee280 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:40:16.679417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f1e0188c-69a6-435f-a828-28f14e8ee280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:40:16.742183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 582/1163] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:40:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:40:17.008627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bb8a5-547d-4cf4-ab80-77b790ff9927 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:17.011627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:17.108744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bb8a5-547d-4cf4-ab80-77b790ff9927 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:17.140914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e31d2b-8f62-49d2-8663-80770333900d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:17.141499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e31d2b-8f62-49d2-8663-80770333900d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:17.141774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e31d2b-8f62-49d2-8663-80770333900d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:17.142038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e31d2b-8f62-49d2-8663-80770333900d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:17.206813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e31d2b-8f62-49d2-8663-80770333900d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:17.208273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e31d2b-8f62-49d2-8663-80770333900d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:17.209981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:17.303104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 582/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:40:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:40:17.308961 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0188c-69a6-435f-a828-28f14e8ee280 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:17.403564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0188c-69a6-435f-a828-28f14e8ee280 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:17.416908 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:17.436857 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2921367-96c3-462c-adf4-00843f0643ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:17.437468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2921367-96c3-462c-adf4-00843f0643ff None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:17.437744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2921367-96c3-462c-adf4-00843f0643ff None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:17.438013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2921367-96c3-462c-adf4-00843f0643ff None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:17.444474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2921367-96c3-462c-adf4-00843f0643ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:17.445899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2921367-96c3-462c-adf4-00843f0643ff None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-e2921367-96c3-462c-adf4-00843f0643ff None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:40:17.457419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:40:17.458626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:40:17.459666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 583/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Aug 18 14:40:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:40:17.464584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e31d2b-8f62-49d2-8663-80770333900d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:17.559379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e31d2b-8f62-49d2-8663-80770333900d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:17.560833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:17.593286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeda00f-45e8-4430-852b-ccfa22cb048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:17.593875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeda00f-45e8-4430-852b-ccfa22cb048e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:17.594156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeda00f-45e8-4430-852b-ccfa22cb048e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:17.594428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeda00f-45e8-4430-852b-ccfa22cb048e None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:17.595062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeda00f-45e8-4430-852b-ccfa22cb048e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:40:17.595366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeda00f-45e8-4430-852b-ccfa22cb048e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:17.596756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeda00f-45e8-4430-852b-ccfa22cb048e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:17.603618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 583/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Aug 18 14:40:17 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:19.465995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5468a1a-81d3-489b-8020-1dd91ebc1474 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:19.466524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5468a1a-81d3-489b-8020-1dd91ebc1474 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:19.466524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5468a1a-81d3-489b-8020-1dd91ebc1474 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:19.467966 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 584/1167] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:40:19.473243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1c2ff-c23b-4c4c-bbbf-976b104002e9 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:19.473645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1c2ff-c23b-4c4c-bbbf-976b104002e9 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:19.473792 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1c2ff-c23b-4c4c-bbbf-976b104002e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:19.491943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-bdb1c2ff-c23b-4c4c-bbbf-976b104002e9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:40:19.493125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-bdb1c2ff-c23b-4c4c-bbbf-976b104002e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:40:19.556744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 584/1168] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:40:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:40:19.832618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5468a1a-81d3-489b-8020-1dd91ebc1474 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:19.930541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5468a1a-81d3-489b-8020-1dd91ebc1474 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:19.963877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7eab5d-2a29-48ef-84f5-0a28da9bba7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:19.964460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7eab5d-2a29-48ef-84f5-0a28da9bba7d None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:19.964733 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7eab5d-2a29-48ef-84f5-0a28da9bba7d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:19.964999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7eab5d-2a29-48ef-84f5-0a28da9bba7d None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:19.965650 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7eab5d-2a29-48ef-84f5-0a28da9bba7d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:19.966992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7eab5d-2a29-48ef-84f5-0a28da9bba7d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:19.981766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 585/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Thu Aug 18 14:40:19 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:40:21.882842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b69cd4-fa1d-43f8-b975-c097efb41e28 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:21.883377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b69cd4-fa1d-43f8-b975-c097efb41e28 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:21.883377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b69cd4-fa1d-43f8-b975-c097efb41e28 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:21.884749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 585/1170] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:40:21.889808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a925f-cdc3-44cb-9c37-1c81fe8a407d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:21.890242 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a925f-cdc3-44cb-9c37-1c81fe8a407d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:21.890343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a925f-cdc3-44cb-9c37-1c81fe8a407d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:21.908185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-2b2a925f-cdc3-44cb-9c37-1c81fe8a407d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:40:21.909355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2b2a925f-cdc3-44cb-9c37-1c81fe8a407d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:40:21.935018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:21.973827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 586/1171] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:40:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:40:22.241811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b69cd4-fa1d-43f8-b975-c097efb41e28 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:22.333179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:22.340141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b69cd4-fa1d-43f8-b975-c097efb41e28 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:22.372741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7985d72-4110-4f3a-92ac-699f5088185a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:22.373898 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7985d72-4110-4f3a-92ac-699f5088185a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:22.374172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7985d72-4110-4f3a-92ac-699f5088185a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:22.374437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7985d72-4110-4f3a-92ac-699f5088185a None admin] PATH_INFO: `/v3/domains/ed2e09bc7ab445e5afa2118c538f157b` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:22.384658 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7985d72-4110-4f3a-92ac-699f5088185a None admin] RBAC: Authorizing `identity:get_domain(domain_id=ed2e09bc7ab445e5afa2118c538f157b)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:22.386103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7985d72-4110-4f3a-92ac-699f5088185a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:22.387652 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 586/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Thu Aug 18 14:40:22 2022] GET /identity/v3/domains/ed2e09bc7ab445e5afa2118c538f157b => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:22.393616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a925f-cdc3-44cb-9c37-1c81fe8a407d None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:22.486173 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a925f-cdc3-44cb-9c37-1c81fe8a407d None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:22.518183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061a085-df35-439b-b7a5-55e367484761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:22.518705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061a085-df35-439b-b7a5-55e367484761 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:22.518978 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061a085-df35-439b-b7a5-55e367484761 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:22.519243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061a085-df35-439b-b7a5-55e367484761 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:22.519895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061a085-df35-439b-b7a5-55e367484761 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:22.521305 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061a085-df35-439b-b7a5-55e367484761 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:22.547012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 587/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:40:22 2022] POST /identity/v3/projects => generated 361 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:40:24.464168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410ffaf-d4fa-4173-9b11-d40c782a6b5d None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:24.464695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410ffaf-d4fa-4173-9b11-d40c782a6b5d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:24.464695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410ffaf-d4fa-4173-9b11-d40c782a6b5d None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:24.466043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 587/1174] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:40:24.471127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43663e27-73f9-43f3-8452-c066c7aecd46 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:24.471552 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43663e27-73f9-43f3-8452-c066c7aecd46 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:24.471671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43663e27-73f9-43f3-8452-c066c7aecd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:24.489525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-43663e27-73f9-43f3-8452-c066c7aecd46 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:40:24.490692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-43663e27-73f9-43f3-8452-c066c7aecd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:40:24.501801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:24.554814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 588/1175] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:40:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:40:24.829215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410ffaf-d4fa-4173-9b11-d40c782a6b5d None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:24.928598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410ffaf-d4fa-4173-9b11-d40c782a6b5d None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:24.962421 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2220be-a436-40dc-af6d-6ff73b782667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:24.963104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2220be-a436-40dc-af6d-6ff73b782667 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:24.963382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2220be-a436-40dc-af6d-6ff73b782667 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:24.963657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2220be-a436-40dc-af6d-6ff73b782667 None admin] PATH_INFO: `/v3/domains/ed2e09bc7ab445e5afa2118c538f157b` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:24.964801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2220be-a436-40dc-af6d-6ff73b782667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ed2e09bc7ab445e5afa2118c538f157b)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:24.966226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2220be-a436-40dc-af6d-6ff73b782667 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:24.967829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 588/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Thu Aug 18 14:40:24 2022] GET /identity/v3/domains/ed2e09bc7ab445e5afa2118c538f157b => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:24.973566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43663e27-73f9-43f3-8452-c066c7aecd46 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:25.065926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43663e27-73f9-43f3-8452-c066c7aecd46 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:25.097898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20550dcc-df10-4194-a66b-925cf42ba5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:25.098425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550dcc-df10-4194-a66b-925cf42ba5da None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:25.098699 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550dcc-df10-4194-a66b-925cf42ba5da None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:25.098969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550dcc-df10-4194-a66b-925cf42ba5da None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:25.099601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20550dcc-df10-4194-a66b-925cf42ba5da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:25.101039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20550dcc-df10-4194-a66b-925cf42ba5da None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:25.121904 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 589/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:40:24 2022] POST /identity/v3/users => generated 311 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:40:27.021893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ee8db-ecc2-4ad9-89d2-8fa7e9ab1d77 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:27.022432 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ee8db-ecc2-4ad9-89d2-8fa7e9ab1d77 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:27.022432 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ee8db-ecc2-4ad9-89d2-8fa7e9ab1d77 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:27.023787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 589/1178] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:40:27.029057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fa72d-88fe-42ee-baa4-c53dee103a55 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:27.029459 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fa72d-88fe-42ee-baa4-c53dee103a55 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:27.029606 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fa72d-88fe-42ee-baa4-c53dee103a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:27.035838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:27.049018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f71fa72d-88fe-42ee-baa4-c53dee103a55 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:40:27.050206 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f71fa72d-88fe-42ee-baa4-c53dee103a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:40:27.113500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 590/1179] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:40:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:40:27.382659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ee8db-ecc2-4ad9-89d2-8fa7e9ab1d77 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:27.408326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:27.481419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ee8db-ecc2-4ad9-89d2-8fa7e9ab1d77 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:27.514129 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c69e34-80eb-4285-949b-88e28a30b8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:27.514746 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c69e34-80eb-4285-949b-88e28a30b8b3 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:27.515025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c69e34-80eb-4285-949b-88e28a30b8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:27.515297 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c69e34-80eb-4285-949b-88e28a30b8b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:27.525472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c69e34-80eb-4285-949b-88e28a30b8b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-c4c69e34-80eb-4285-949b-88e28a30b8b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:40:27.530122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:40:27.531224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:40:27.532475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 590/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:40:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:40:27.537113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fa72d-88fe-42ee-baa4-c53dee103a55 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:27.631177 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fa72d-88fe-42ee-baa4-c53dee103a55 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:27.659037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:27.664208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7586e-40d5-4b20-aca0-9000eb7ef507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:27.664797 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7586e-40d5-4b20-aca0-9000eb7ef507 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:27.665074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7586e-40d5-4b20-aca0-9000eb7ef507 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:27.665351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7586e-40d5-4b20-aca0-9000eb7ef507 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:27.666003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7586e-40d5-4b20-aca0-9000eb7ef507 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:40:27.666300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7586e-40d5-4b20-aca0-9000eb7ef507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:27.667658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7586e-40d5-4b20-aca0-9000eb7ef507 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:27.675892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 591/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:40:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:27.681509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c69e34-80eb-4285-949b-88e28a30b8b3 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:27.763760 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:27.779244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c69e34-80eb-4285-949b-88e28a30b8b3 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:27.812007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabea314-3b2e-4303-b1b1-a565aaf0a3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:27.812618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabea314-3b2e-4303-b1b1-a565aaf0a3c8 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:27.812895 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabea314-3b2e-4303-b1b1-a565aaf0a3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:27.813165 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabea314-3b2e-4303-b1b1-a565aaf0a3c8 None admin] PATH_INFO: `/v3/users/bad92c911e9c42c6b1fd71a821692285` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:27.826463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabea314-3b2e-4303-b1b1-a565aaf0a3c8 None admin] RBAC: Authorizing `identity:get_user(user_id=bad92c911e9c42c6b1fd71a821692285)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:27.827935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabea314-3b2e-4303-b1b1-a565aaf0a3c8 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:27.829548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 591/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:40:27 2022] GET /identity/v3/users/bad92c911e9c42c6b1fd71a821692285 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:27.835463 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7586e-40d5-4b20-aca0-9000eb7ef507 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:27.934271 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7586e-40d5-4b20-aca0-9000eb7ef507 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:27.966479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d3878-f395-4ff1-b591-0e56a96f698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:27.967091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d3878-f395-4ff1-b591-0e56a96f698e None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:27.967366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d3878-f395-4ff1-b591-0e56a96f698e None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:27.967632 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d3878-f395-4ff1-b591-0e56a96f698e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:28.025392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:28.033188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d3878-f395-4ff1-b591-0e56a96f698e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:28.034634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d3878-f395-4ff1-b591-0e56a96f698e None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:28.131256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 592/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:40:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:40:28.137122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabea314-3b2e-4303-b1b1-a565aaf0a3c8 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:28.232724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabea314-3b2e-4303-b1b1-a565aaf0a3c8 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:28.265367 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715e6c8-c44e-447f-8fa7-5a65cf25f6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:28.265973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715e6c8-c44e-447f-8fa7-5a65cf25f6f8 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:28.266257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715e6c8-c44e-447f-8fa7-5a65cf25f6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:28.266528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715e6c8-c44e-447f-8fa7-5a65cf25f6f8 None admin] PATH_INFO: `/v3/projects/112cd839af3345feb4ed8cec4f43c244` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:28.267627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715e6c8-c44e-447f-8fa7-5a65cf25f6f8 None admin] RBAC: Authorizing `identity:get_project(project_id=112cd839af3345feb4ed8cec4f43c244)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:28.269069 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715e6c8-c44e-447f-8fa7-5a65cf25f6f8 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:28.270653 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 592/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:40:28 2022] GET /identity/v3/projects/112cd839af3345feb4ed8cec4f43c244 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:28.276721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d3878-f395-4ff1-b591-0e56a96f698e None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:28.364605 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:28.371103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d3878-f395-4ff1-b591-0e56a96f698e None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:28.403540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65212cec-ced7-4242-a5c1-b7278c8639b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:28.404156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65212cec-ced7-4242-a5c1-b7278c8639b8 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:28.404429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65212cec-ced7-4242-a5c1-b7278c8639b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:28.404698 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65212cec-ced7-4242-a5c1-b7278c8639b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:28.405407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65212cec-ced7-4242-a5c1-b7278c8639b8 None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=112cd839af3345feb4ed8cec4f43c244, user.id=bad92c911e9c42c6b1fd71a821692285) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:40:28.405713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65212cec-ced7-4242-a5c1-b7278c8639b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:28.407118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65212cec-ced7-4242-a5c1-b7278c8639b8 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:28.412262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 593/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:40:28 2022] GET /identity/v3/role_assignments?user.id=bad92c911e9c42c6b1fd71a821692285&scope.project.id=112cd839af3345feb4ed8cec4f43c244&role.id=9a439cf8eef34ada85428b378878e63e => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:30.261451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec21bd7-05f0-430c-9f2c-cb7612e06af9 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:30.261995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec21bd7-05f0-430c-9f2c-cb7612e06af9 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:30.261995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec21bd7-05f0-430c-9f2c-cb7612e06af9 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:30.263314 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 593/1186] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:40:30.268508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad02ed7-875e-4b6a-be5c-6dfe47de3a12 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:30.268911 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad02ed7-875e-4b6a-be5c-6dfe47de3a12 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:30.269061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad02ed7-875e-4b6a-be5c-6dfe47de3a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:30.287177 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-5ad02ed7-875e-4b6a-be5c-6dfe47de3a12 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:40:30.288404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5ad02ed7-875e-4b6a-be5c-6dfe47de3a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:40:30.351942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 594/1187] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:40:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:40:30.614566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec21bd7-05f0-430c-9f2c-cb7612e06af9 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:30.626936 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:30.714112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec21bd7-05f0-430c-9f2c-cb7612e06af9 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:30.748786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d22b8-dedb-456f-a7ad-a7c8f49967d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:30.749403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d22b8-dedb-456f-a7ad-a7c8f49967d4 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:30.749682 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d22b8-dedb-456f-a7ad-a7c8f49967d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:30.749956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d22b8-dedb-456f-a7ad-a7c8f49967d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:30.760073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d22b8-dedb-456f-a7ad-a7c8f49967d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-e27d22b8-dedb-456f-a7ad-a7c8f49967d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:40:30.765258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:40:30.766329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:40:30.767537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 594/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:40:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:40:30.772331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad02ed7-875e-4b6a-be5c-6dfe47de3a12 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:30.854962 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:30.866634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad02ed7-875e-4b6a-be5c-6dfe47de3a12 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:30.899709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c40ce-dee9-4119-9a9c-b919bdc29247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:30.900324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c40ce-dee9-4119-9a9c-b919bdc29247 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:30.900610 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c40ce-dee9-4119-9a9c-b919bdc29247 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:30.900884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c40ce-dee9-4119-9a9c-b919bdc29247 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:30.901543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c40ce-dee9-4119-9a9c-b919bdc29247 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:40:30.901846 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c40ce-dee9-4119-9a9c-b919bdc29247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:30.903208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c40ce-dee9-4119-9a9c-b919bdc29247 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:30.911056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 595/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:40:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:30.916867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d22b8-dedb-456f-a7ad-a7c8f49967d4 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:30.984518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:31.015877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d22b8-dedb-456f-a7ad-a7c8f49967d4 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:31.048109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae971a8-eaa5-43a2-be3e-e7b447f9b754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:31.048720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae971a8-eaa5-43a2-be3e-e7b447f9b754 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:31.048996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae971a8-eaa5-43a2-be3e-e7b447f9b754 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:31.049267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae971a8-eaa5-43a2-be3e-e7b447f9b754 None admin] PATH_INFO: `/v3/users/bad92c911e9c42c6b1fd71a821692285` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:31.050370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae971a8-eaa5-43a2-be3e-e7b447f9b754 None admin] RBAC: Authorizing `identity:get_user(user_id=bad92c911e9c42c6b1fd71a821692285)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:31.051824 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae971a8-eaa5-43a2-be3e-e7b447f9b754 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:31.053376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 595/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:40:30 2022] GET /identity/v3/users/bad92c911e9c42c6b1fd71a821692285 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:31.059076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c40ce-dee9-4119-9a9c-b919bdc29247 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:31.155554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c40ce-dee9-4119-9a9c-b919bdc29247 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:31.187409 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601db04-f314-4c3d-9ac8-80b94ebfabd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:31.188056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601db04-f314-4c3d-9ac8-80b94ebfabd2 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:31.188401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601db04-f314-4c3d-9ac8-80b94ebfabd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:31.188693 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601db04-f314-4c3d-9ac8-80b94ebfabd2 None admin] PATH_INFO: `/v3/projects/112cd839af3345feb4ed8cec4f43c244` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:31.190175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601db04-f314-4c3d-9ac8-80b94ebfabd2 None admin] RBAC: Authorizing `identity:get_project(project_id=112cd839af3345feb4ed8cec4f43c244)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:31.191624 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601db04-f314-4c3d-9ac8-80b94ebfabd2 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:31.193284 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 596/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:40:31 2022] GET /identity/v3/projects/112cd839af3345feb4ed8cec4f43c244 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:31.199189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae971a8-eaa5-43a2-be3e-e7b447f9b754 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:31.294456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae971a8-eaa5-43a2-be3e-e7b447f9b754 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:31.326798 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad91ad-9ea9-4a50-bc8b-18df18753d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:31.327437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad91ad-9ea9-4a50-bc8b-18df18753d23 None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:31.327742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad91ad-9ea9-4a50-bc8b-18df18753d23 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:31.328012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad91ad-9ea9-4a50-bc8b-18df18753d23 None admin] PATH_INFO: `/v3/projects/112cd839af3345feb4ed8cec4f43c244/users/bad92c911e9c42c6b1fd71a821692285/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:31.329878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad91ad-9ea9-4a50-bc8b-18df18753d23 None admin] RBAC: Authorizing `identity:create_grant(project_id=112cd839af3345feb4ed8cec4f43c244, user_id=bad92c911e9c42c6b1fd71a821692285, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:31.331296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad91ad-9ea9-4a50-bc8b-18df18753d23 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:31.338884 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:31.345205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 596/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:40:31 2022] PUT /identity/v3/projects/112cd839af3345feb4ed8cec4f43c244/users/bad92c911e9c42c6b1fd71a821692285/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:40:33.199975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fdbd6-1c0c-4816-a38d-53fa8b833f97 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:33.200545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fdbd6-1c0c-4816-a38d-53fa8b833f97 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:33.200545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fdbd6-1c0c-4816-a38d-53fa8b833f97 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:33.202365 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 597/1193] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:40:33.207452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59690e5d-8ff2-499c-a0d2-1ca2395955f0 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:33.207890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59690e5d-8ff2-499c-a0d2-1ca2395955f0 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:33.208029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59690e5d-8ff2-499c-a0d2-1ca2395955f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:33.226124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-59690e5d-8ff2-499c-a0d2-1ca2395955f0 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:40:33.227324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-59690e5d-8ff2-499c-a0d2-1ca2395955f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:40:33.290377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 597/1194] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:40:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:40:33.554844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fdbd6-1c0c-4816-a38d-53fa8b833f97 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:33.573516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:33.651876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fdbd6-1c0c-4816-a38d-53fa8b833f97 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:33.683387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b82500-f03c-4d29-944d-fa531d1c0164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:33.684019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b82500-f03c-4d29-944d-fa531d1c0164 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:33.684292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b82500-f03c-4d29-944d-fa531d1c0164 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:33.684557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b82500-f03c-4d29-944d-fa531d1c0164 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:33.694074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b82500-f03c-4d29-944d-fa531d1c0164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-62b82500-f03c-4d29-944d-fa531d1c0164 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:40:33.698824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:40:33.699997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:40:33.701688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 598/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:40:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:40:33.706450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59690e5d-8ff2-499c-a0d2-1ca2395955f0 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:33.801091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59690e5d-8ff2-499c-a0d2-1ca2395955f0 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:33.825757 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:33.833996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d8e6c-e829-4a06-bec9-159844ec8714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:33.834588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d8e6c-e829-4a06-bec9-159844ec8714 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:33.834862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d8e6c-e829-4a06-bec9-159844ec8714 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:33.835130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d8e6c-e829-4a06-bec9-159844ec8714 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:33.835816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d8e6c-e829-4a06-bec9-159844ec8714 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:40:33.836116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d8e6c-e829-4a06-bec9-159844ec8714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:33.837472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d8e6c-e829-4a06-bec9-159844ec8714 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:33.845219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 598/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:40:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:33.850864 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b82500-f03c-4d29-944d-fa531d1c0164 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:33.926484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:33.948079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b82500-f03c-4d29-944d-fa531d1c0164 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:33.980433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a029c5b-bcd8-48c2-bd8d-1305fe4bac0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:33.981050 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a029c5b-bcd8-48c2-bd8d-1305fe4bac0a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:33.981325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a029c5b-bcd8-48c2-bd8d-1305fe4bac0a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:33.981596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a029c5b-bcd8-48c2-bd8d-1305fe4bac0a None admin] PATH_INFO: `/v3/users/bad92c911e9c42c6b1fd71a821692285` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:33.982695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a029c5b-bcd8-48c2-bd8d-1305fe4bac0a None admin] RBAC: Authorizing `identity:get_user(user_id=bad92c911e9c42c6b1fd71a821692285)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:33.984221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a029c5b-bcd8-48c2-bd8d-1305fe4bac0a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:33.985781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 599/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Aug 18 14:40:33 2022] GET /identity/v3/users/bad92c911e9c42c6b1fd71a821692285 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:33.991643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d8e6c-e829-4a06-bec9-159844ec8714 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:34.088200 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d8e6c-e829-4a06-bec9-159844ec8714 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:34.121681 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e203af-a096-4155-b148-9339bc00131c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:34.122274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e203af-a096-4155-b148-9339bc00131c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:34.122550 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e203af-a096-4155-b148-9339bc00131c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:34.123338 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e203af-a096-4155-b148-9339bc00131c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:34.177807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:34.188927 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e203af-a096-4155-b148-9339bc00131c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:34.190356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e203af-a096-4155-b148-9339bc00131c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:34.283438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 599/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:40:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:40:34.289257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a029c5b-bcd8-48c2-bd8d-1305fe4bac0a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:34.383745 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a029c5b-bcd8-48c2-bd8d-1305fe4bac0a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:34.414599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ca885-d5c5-4f5c-8c60-6731b143bf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:34.415200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ca885-d5c5-4f5c-8c60-6731b143bf10 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:34.415472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ca885-d5c5-4f5c-8c60-6731b143bf10 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:34.415765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ca885-d5c5-4f5c-8c60-6731b143bf10 None admin] PATH_INFO: `/v3/projects/112cd839af3345feb4ed8cec4f43c244` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:34.416858 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ca885-d5c5-4f5c-8c60-6731b143bf10 None admin] RBAC: Authorizing `identity:get_project(project_id=112cd839af3345feb4ed8cec4f43c244)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:34.418259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ca885-d5c5-4f5c-8c60-6731b143bf10 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:34.420373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 600/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Aug 18 14:40:34 2022] GET /identity/v3/projects/112cd839af3345feb4ed8cec4f43c244 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:34.426223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e203af-a096-4155-b148-9339bc00131c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:40:34.512621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:34.522312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e203af-a096-4155-b148-9339bc00131c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:40:34.554873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffa221-ceb5-45a7-ba0e-2b19c84bd7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:40:34.555460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffa221-ceb5-45a7-ba0e-2b19c84bd7e5 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:40:34.555764 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffa221-ceb5-45a7-ba0e-2b19c84bd7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:40:34.556039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffa221-ceb5-45a7-ba0e-2b19c84bd7e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:40:34.556751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffa221-ceb5-45a7-ba0e-2b19c84bd7e5 None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=112cd839af3345feb4ed8cec4f43c244, user.id=bad92c911e9c42c6b1fd71a821692285) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:40:34.557056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffa221-ceb5-45a7-ba0e-2b19c84bd7e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:40:34.558472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffa221-ceb5-45a7-ba0e-2b19c84bd7e5 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:40:34.564680 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 600/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:40:34 2022] GET /identity/v3/role_assignments?user.id=bad92c911e9c42c6b1fd71a821692285&scope.project.id=112cd839af3345feb4ed8cec4f43c244&role.id=9a439cf8eef34ada85428b378878e63e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:40:44.410673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=93 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:40:44.562158 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=23 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:29.200942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05829083-4c1c-4255-903e-c22836124eda None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:29.201533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05829083-4c1c-4255-903e-c22836124eda None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:29.201651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05829083-4c1c-4255-903e-c22836124eda None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:29.202916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 601/1201] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:41:29.210332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bd434-bca2-42df-a9c9-c3e8a2c5d88c None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:29.210847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bd434-bca2-42df-a9c9-c3e8a2c5d88c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:29.210847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bd434-bca2-42df-a9c9-c3e8a2c5d88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:29.229297 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-c67bd434-bca2-42df-a9c9-c3e8a2c5d88c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:41:29.230442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c67bd434-bca2-42df-a9c9-c3e8a2c5d88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:41:29.322769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 601/1202] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:41:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:41:29.618321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05829083-4c1c-4255-903e-c22836124eda None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:29.737231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05829083-4c1c-4255-903e-c22836124eda None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:29.773969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623e4b2-3a39-4ec5-a0e8-aa0de78db4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:29.774594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623e4b2-3a39-4ec5-a0e8-aa0de78db4fd None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:29.774884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623e4b2-3a39-4ec5-a0e8-aa0de78db4fd None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:29.775156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623e4b2-3a39-4ec5-a0e8-aa0de78db4fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:29.786611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623e4b2-3a39-4ec5-a0e8-aa0de78db4fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:29.788160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623e4b2-3a39-4ec5-a0e8-aa0de78db4fd None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-b623e4b2-3a39-4ec5-a0e8-aa0de78db4fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:29.808154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:29.811890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:29.812965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:29.812965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:29.812965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:29.812965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 602/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:29.816542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67bd434-bca2-42df-a9c9-c3e8a2c5d88c None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:29.951630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67bd434-bca2-42df-a9c9-c3e8a2c5d88c None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:29.990997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:29.999437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b737a-7714-43ef-9b52-8ff44756afa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:30.000070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b737a-7714-43ef-9b52-8ff44756afa9 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:30.000350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b737a-7714-43ef-9b52-8ff44756afa9 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:30.000623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b737a-7714-43ef-9b52-8ff44756afa9 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:30.001276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b737a-7714-43ef-9b52-8ff44756afa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:30.002243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b737a-7714-43ef-9b52-8ff44756afa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:30.004039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b737a-7714-43ef-9b52-8ff44756afa9 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:30.021456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 602/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:30.032116 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623e4b2-3a39-4ec5-a0e8-aa0de78db4fd None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:30.102065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:30.152722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623e4b2-3a39-4ec5-a0e8-aa0de78db4fd None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:30.191145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f68ab-8208-498b-8dc6-5ba68726204b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:30.191935 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f68ab-8208-498b-8dc6-5ba68726204b None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:30.192329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f68ab-8208-498b-8dc6-5ba68726204b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:30.192700 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f68ab-8208-498b-8dc6-5ba68726204b None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:30.193453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f68ab-8208-498b-8dc6-5ba68726204b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:30.194979 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f68ab-8208-498b-8dc6-5ba68726204b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:30.223806 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 603/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Aug 18 14:41:30 2022] POST /identity/v3/users => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:41:32.329599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eace9-368b-4b57-ad87-a3bfa042f246 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:32.330172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eace9-368b-4b57-ad87-a3bfa042f246 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:32.330172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eace9-368b-4b57-ad87-a3bfa042f246 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:32.332021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 603/1206] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:41:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:41:32.338561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180ca81-bd07-4d53-aab8-ad9dcaebb6a2 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:32.339108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180ca81-bd07-4d53-aab8-ad9dcaebb6a2 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:32.339108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180ca81-bd07-4d53-aab8-ad9dcaebb6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:32.364167 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f180ca81-bd07-4d53-aab8-ad9dcaebb6a2 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:41:32.365540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f180ca81-bd07-4d53-aab8-ad9dcaebb6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:41:32.446524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 604/1207] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:41:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:41:32.716036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eace9-368b-4b57-ad87-a3bfa042f246 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:32.838095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eace9-368b-4b57-ad87-a3bfa042f246 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:32.878079 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488a2a0-7b8c-44e8-9a8e-033d6595dcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:32.878715 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488a2a0-7b8c-44e8-9a8e-033d6595dcfc None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:32.878989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488a2a0-7b8c-44e8-9a8e-033d6595dcfc None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:32.879265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488a2a0-7b8c-44e8-9a8e-033d6595dcfc None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:32.897420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4488a2a0-7b8c-44e8-9a8e-033d6595dcfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-4488a2a0-7b8c-44e8-9a8e-033d6595dcfc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:41:32.902577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:32.907769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:32.908920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 604/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:41:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:32.910621 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180ca81-bd07-4d53-aab8-ad9dcaebb6a2 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:32.955525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:33.017918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180ca81-bd07-4d53-aab8-ad9dcaebb6a2 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:33.055044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbabc59-9efc-46c9-9c67-ba227380d8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:33.055822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbabc59-9efc-46c9-9c67-ba227380d8f8 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:33.056229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbabc59-9efc-46c9-9c67-ba227380d8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:33.056516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbabc59-9efc-46c9-9c67-ba227380d8f8 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:33.057311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbabc59-9efc-46c9-9c67-ba227380d8f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:33.057627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbabc59-9efc-46c9-9c67-ba227380d8f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:33.059423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbabc59-9efc-46c9-9c67-ba227380d8f8 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:33.068454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 605/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:41:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:33.074346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488a2a0-7b8c-44e8-9a8e-033d6595dcfc None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:33.131033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:33.188458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488a2a0-7b8c-44e8-9a8e-033d6595dcfc None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:33.227522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a82ed-75a6-4312-b688-5bd19c134270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:33.228180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a82ed-75a6-4312-b688-5bd19c134270 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:33.228463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a82ed-75a6-4312-b688-5bd19c134270 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:33.228845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a82ed-75a6-4312-b688-5bd19c134270 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:33.239972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a82ed-75a6-4312-b688-5bd19c134270 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:33.241407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a82ed-75a6-4312-b688-5bd19c134270 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-a95a82ed-75a6-4312-b688-5bd19c134270 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:33.261160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:33.262265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:33.263361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:33.263361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:33.263361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:33.263361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 605/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:33.268710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbabc59-9efc-46c9-9c67-ba227380d8f8 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:33.380561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:33.383610 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbabc59-9efc-46c9-9c67-ba227380d8f8 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:33.427007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5163c2-fb19-4d5b-a3a3-e6283364100d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:33.427751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5163c2-fb19-4d5b-a3a3-e6283364100d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:33.427954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5163c2-fb19-4d5b-a3a3-e6283364100d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:33.428195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5163c2-fb19-4d5b-a3a3-e6283364100d None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:33.428838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5163c2-fb19-4d5b-a3a3-e6283364100d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:33.429147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5163c2-fb19-4d5b-a3a3-e6283364100d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:33.430503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5163c2-fb19-4d5b-a3a3-e6283364100d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:33.443342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 606/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:41:33.450029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a82ed-75a6-4312-b688-5bd19c134270 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:33.569957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a82ed-75a6-4312-b688-5bd19c134270 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:33.571595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:33.606760 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42478cf-8266-4db1-88b5-ceb22d3bab77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:33.607395 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42478cf-8266-4db1-88b5-ceb22d3bab77 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:33.607673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42478cf-8266-4db1-88b5-ceb22d3bab77 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:33.608909 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42478cf-8266-4db1-88b5-ceb22d3bab77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:33.616442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42478cf-8266-4db1-88b5-ceb22d3bab77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:33.618225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42478cf-8266-4db1-88b5-ceb22d3bab77 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-f42478cf-8266-4db1-88b5-ceb22d3bab77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:41:33.631159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:33.632415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:33.633572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:33.633572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:33.633572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:41:33.633572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:41:33.633572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:41:33.633572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:41:33.633572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:41:33.633572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:41:33.633572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 14:41:33.633572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:33.635842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 606/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:33.638303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5163c2-fb19-4d5b-a3a3-e6283364100d None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:33.758360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5163c2-fb19-4d5b-a3a3-e6283364100d None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:33.801874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcb484-95b3-4907-921c-f2b7d6f2a42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:33.801874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcb484-95b3-4907-921c-f2b7d6f2a42d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:33.802462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcb484-95b3-4907-921c-f2b7d6f2a42d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:33.802746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcb484-95b3-4907-921c-f2b7d6f2a42d None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:33.803879 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcb484-95b3-4907-921c-f2b7d6f2a42d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:33.804199 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcb484-95b3-4907-921c-f2b7d6f2a42d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:33.807110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcb484-95b3-4907-921c-f2b7d6f2a42d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:33.832143 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 607/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Thu Aug 18 14:41:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:33.839223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42478cf-8266-4db1-88b5-ceb22d3bab77 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:33.975901 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42478cf-8266-4db1-88b5-ceb22d3bab77 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:34.019220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f59d8-33c1-4691-a873-fda373689961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:34.020003 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f59d8-33c1-4691-a873-fda373689961 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:34.020277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f59d8-33c1-4691-a873-fda373689961 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:34.020546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f59d8-33c1-4691-a873-fda373689961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:34.052272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:34.094511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f59d8-33c1-4691-a873-fda373689961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:34.096348 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f59d8-33c1-4691-a873-fda373689961 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:34.220665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 607/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:41:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:41:34.226415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcb484-95b3-4907-921c-f2b7d6f2a42d None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:34.245455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:34.347260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcb484-95b3-4907-921c-f2b7d6f2a42d None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:34.401895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60ebb6-2a82-4555-815e-fdd6afcd0e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:34.402800 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60ebb6-2a82-4555-815e-fdd6afcd0e39 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:34.403824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60ebb6-2a82-4555-815e-fdd6afcd0e39 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:34.404292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60ebb6-2a82-4555-815e-fdd6afcd0e39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:34.417658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60ebb6-2a82-4555-815e-fdd6afcd0e39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:34.419253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60ebb6-2a82-4555-815e-fdd6afcd0e39 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-ab60ebb6-2a82-4555-815e-fdd6afcd0e39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:34.441103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:34.442391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:34.443481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:34.443481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:34.443481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:34.443481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 608/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:34.448995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f59d8-33c1-4691-a873-fda373689961 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:34.513273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:34.554368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f59d8-33c1-4691-a873-fda373689961 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:34.592219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b1556-f493-4637-9865-620466ce5cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:34.592988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b1556-f493-4637-9865-620466ce5cf9 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:34.593411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b1556-f493-4637-9865-620466ce5cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:34.593749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b1556-f493-4637-9865-620466ce5cf9 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:34.594568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b1556-f493-4637-9865-620466ce5cf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:34.594935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b1556-f493-4637-9865-620466ce5cf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:34.596394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b1556-f493-4637-9865-620466ce5cf9 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:34.608430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 608/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:34.614607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60ebb6-2a82-4555-815e-fdd6afcd0e39 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:34.693301 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:34.729539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60ebb6-2a82-4555-815e-fdd6afcd0e39 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:34.768109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd62b4c-2055-473f-ba95-9e43298c4eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:34.769095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd62b4c-2055-473f-ba95-9e43298c4eff None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:34.769546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd62b4c-2055-473f-ba95-9e43298c4eff None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:34.769984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd62b4c-2055-473f-ba95-9e43298c4eff None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:34.777473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd62b4c-2055-473f-ba95-9e43298c4eff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:34.779241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd62b4c-2055-473f-ba95-9e43298c4eff None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-7dd62b4c-2055-473f-ba95-9e43298c4eff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:41:34.791583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:34.792751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:34.793872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:41:34.793872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:41:34.793872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:41:34.793872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:41:34.793872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:41:34.793872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:41:34.793872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:41:34.793872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:34.793872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 609/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:41:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:34.800608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b1556-f493-4637-9865-620466ce5cf9 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:34.916212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b1556-f493-4637-9865-620466ce5cf9 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:34.949673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:34.963168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495da43-7a90-4f01-8ae5-da122a46084e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:34.964127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495da43-7a90-4f01-8ae5-da122a46084e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:34.964622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495da43-7a90-4f01-8ae5-da122a46084e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:34.965055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495da43-7a90-4f01-8ae5-da122a46084e None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:34.965945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495da43-7a90-4f01-8ae5-da122a46084e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:34.966408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495da43-7a90-4f01-8ae5-da122a46084e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:34.968548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495da43-7a90-4f01-8ae5-da122a46084e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:34.983221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 609/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:41:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:34.989611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd62b4c-2055-473f-ba95-9e43298c4eff None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:35.111427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd62b4c-2055-473f-ba95-9e43298c4eff None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:35.142515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:35.149274 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bebed5-0e6e-43a9-b8ef-61de77356c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:35.149892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bebed5-0e6e-43a9-b8ef-61de77356c82 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:35.150207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bebed5-0e6e-43a9-b8ef-61de77356c82 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:35.150485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bebed5-0e6e-43a9-b8ef-61de77356c82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:35.151248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bebed5-0e6e-43a9-b8ef-61de77356c82 None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=db581b10d78a4f9eae33fadce0d92b0d) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:35.151558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bebed5-0e6e-43a9-b8ef-61de77356c82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:35.153076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bebed5-0e6e-43a9-b8ef-61de77356c82 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:35.158454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 610/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:41:34 2022] GET /identity/v3/role_assignments?user.id=db581b10d78a4f9eae33fadce0d92b0d&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:37.040949 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3889b0a-fcb8-4615-9341-33f63c80272b None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:37.041559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3889b0a-fcb8-4615-9341-33f63c80272b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:37.041559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3889b0a-fcb8-4615-9341-33f63c80272b None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:37.042889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 610/1220] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:41:37.048933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5e40e-863a-482c-8cec-79340c687e9b None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:37.050277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5e40e-863a-482c-8cec-79340c687e9b None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:37.050277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5e40e-863a-482c-8cec-79340c687e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:37.068306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-7df5e40e-863a-482c-8cec-79340c687e9b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:41:37.069516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7df5e40e-863a-482c-8cec-79340c687e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:41:37.140610 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 611/1221] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:41:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:41:37.408574 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3889b0a-fcb8-4615-9341-33f63c80272b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:37.513101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3889b0a-fcb8-4615-9341-33f63c80272b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:37.547980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7015dc-869f-4840-8d6f-0ddd8fcbb68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:37.548606 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7015dc-869f-4840-8d6f-0ddd8fcbb68c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:37.548882 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7015dc-869f-4840-8d6f-0ddd8fcbb68c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:37.549151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7015dc-869f-4840-8d6f-0ddd8fcbb68c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:37.559581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7015dc-869f-4840-8d6f-0ddd8fcbb68c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-cf7015dc-869f-4840-8d6f-0ddd8fcbb68c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:41:37.564414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:37.565564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:37.566435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 611/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:41:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:37.571551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5e40e-863a-482c-8cec-79340c687e9b None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:37.681362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5e40e-863a-482c-8cec-79340c687e9b None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:37.706877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:37.717555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92030b6e-a3b9-49a0-ac10-514704bdc5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:37.718151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92030b6e-a3b9-49a0-ac10-514704bdc5a8 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:37.718424 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92030b6e-a3b9-49a0-ac10-514704bdc5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:37.718691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92030b6e-a3b9-49a0-ac10-514704bdc5a8 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:37.719344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92030b6e-a3b9-49a0-ac10-514704bdc5a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:37.719641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92030b6e-a3b9-49a0-ac10-514704bdc5a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:37.721033 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92030b6e-a3b9-49a0-ac10-514704bdc5a8 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:37.729349 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 612/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:41:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:37.735152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7015dc-869f-4840-8d6f-0ddd8fcbb68c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:37.781555 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:37.845149 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7015dc-869f-4840-8d6f-0ddd8fcbb68c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:37.882646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544d35-78c4-4363-8a88-4585596fc79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:37.883385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544d35-78c4-4363-8a88-4585596fc79a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:37.883639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544d35-78c4-4363-8a88-4585596fc79a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:37.883962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544d35-78c4-4363-8a88-4585596fc79a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:37.895754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d544d35-78c4-4363-8a88-4585596fc79a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:37.897409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d544d35-78c4-4363-8a88-4585596fc79a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-9d544d35-78c4-4363-8a88-4585596fc79a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:37.912151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:37.913371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:37.914341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:37.914341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:37.914341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:37.914341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 612/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:37.920971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92030b6e-a3b9-49a0-ac10-514704bdc5a8 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:38.026551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92030b6e-a3b9-49a0-ac10-514704bdc5a8 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:38.065615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7045805-8ecc-423a-806c-dc8124411cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:38.066820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7045805-8ecc-423a-806c-dc8124411cf5 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:38.067263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7045805-8ecc-423a-806c-dc8124411cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:38.067645 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7045805-8ecc-423a-806c-dc8124411cf5 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:38.068816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7045805-8ecc-423a-806c-dc8124411cf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:38.069364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7045805-8ecc-423a-806c-dc8124411cf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:38.071100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7045805-8ecc-423a-806c-dc8124411cf5 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:38.090474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 613/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:41:38.092027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544d35-78c4-4363-8a88-4585596fc79a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:38.202708 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:38.210125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544d35-78c4-4363-8a88-4585596fc79a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:38.248269 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead06f2-400d-4f95-b4fa-fe9ebb21d769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:38.249019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead06f2-400d-4f95-b4fa-fe9ebb21d769 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:38.249342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead06f2-400d-4f95-b4fa-fe9ebb21d769 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:38.249659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead06f2-400d-4f95-b4fa-fe9ebb21d769 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:38.258021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead06f2-400d-4f95-b4fa-fe9ebb21d769 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:38.259497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead06f2-400d-4f95-b4fa-fe9ebb21d769 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-fead06f2-400d-4f95-b4fa-fe9ebb21d769 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:41:38.272319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:38.273459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:38.274407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:38.274407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:38.274407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:41:38.274407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:41:38.274407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:41:38.274407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:41:38.274407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:41:38.274407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:41:38.274407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 14:41:38.274407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:38.274967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 613/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:38.280025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7045805-8ecc-423a-806c-dc8124411cf5 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:38.350349 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:38.414038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7045805-8ecc-423a-806c-dc8124411cf5 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:38.451080 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58031cea-4f22-404b-ae28-29d905b5f74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:38.451704 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58031cea-4f22-404b-ae28-29d905b5f74a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:38.451980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58031cea-4f22-404b-ae28-29d905b5f74a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:38.452295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58031cea-4f22-404b-ae28-29d905b5f74a None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:38.452964 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58031cea-4f22-404b-ae28-29d905b5f74a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:38.453264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58031cea-4f22-404b-ae28-29d905b5f74a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:38.454699 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58031cea-4f22-404b-ae28-29d905b5f74a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:38.479291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 614/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Thu Aug 18 14:41:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:38.480604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead06f2-400d-4f95-b4fa-fe9ebb21d769 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:38.592058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead06f2-400d-4f95-b4fa-fe9ebb21d769 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:38.626527 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026256f-6bac-409c-9349-7da4dff05995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:38.627534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026256f-6bac-409c-9349-7da4dff05995 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:38.627946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026256f-6bac-409c-9349-7da4dff05995 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:38.628316 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026256f-6bac-409c-9349-7da4dff05995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:38.640755 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026256f-6bac-409c-9349-7da4dff05995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:38.642205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026256f-6bac-409c-9349-7da4dff05995 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-8026256f-6bac-409c-9349-7da4dff05995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:38.657185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:38.660490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:38.661526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:38.661526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:38.661526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:38.661526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 614/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:41:38.664429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58031cea-4f22-404b-ae28-29d905b5f74a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:38.776861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58031cea-4f22-404b-ae28-29d905b5f74a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:38.779308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:38.815257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6908ef9-7b18-422e-80f4-6ff9c25fa62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:38.815886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6908ef9-7b18-422e-80f4-6ff9c25fa62a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:38.816162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6908ef9-7b18-422e-80f4-6ff9c25fa62a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:38.816467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6908ef9-7b18-422e-80f4-6ff9c25fa62a None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:38.817063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6908ef9-7b18-422e-80f4-6ff9c25fa62a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:38.817367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6908ef9-7b18-422e-80f4-6ff9c25fa62a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:38.818713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6908ef9-7b18-422e-80f4-6ff9c25fa62a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:38.830580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 615/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:41:38.836461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026256f-6bac-409c-9349-7da4dff05995 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:38.846419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:38.951654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026256f-6bac-409c-9349-7da4dff05995 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:38.994448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f5e4e-9208-4985-b9a6-d573a4342042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:38.995243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f5e4e-9208-4985-b9a6-d573a4342042 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:38.995530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f5e4e-9208-4985-b9a6-d573a4342042 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:38.995884 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f5e4e-9208-4985-b9a6-d573a4342042 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:39.004156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f5e4e-9208-4985-b9a6-d573a4342042 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:39.005992 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f5e4e-9208-4985-b9a6-d573a4342042 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-bb3f5e4e-9208-4985-b9a6-d573a4342042 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:41:39.019084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:39.020199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:39.021305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:41:39.021305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:41:39.021305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:41:39.021305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:41:39.021305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:41:39.021305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:41:39.021305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:41:39.021305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:39.021305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 615/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:41:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:39.026460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6908ef9-7b18-422e-80f4-6ff9c25fa62a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:39.138799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6908ef9-7b18-422e-80f4-6ff9c25fa62a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:39.175781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109bfa9-0f73-433d-be1a-12f87a9cdf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:39.176435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109bfa9-0f73-433d-be1a-12f87a9cdf66 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:39.176721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109bfa9-0f73-433d-be1a-12f87a9cdf66 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:39.176948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109bfa9-0f73-433d-be1a-12f87a9cdf66 None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:39.177763 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109bfa9-0f73-433d-be1a-12f87a9cdf66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:39.178053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109bfa9-0f73-433d-be1a-12f87a9cdf66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:39.179613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109bfa9-0f73-433d-be1a-12f87a9cdf66 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:39.192955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 616/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:41:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:41:39.199382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f5e4e-9208-4985-b9a6-d573a4342042 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:39.271928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:39.313589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f5e4e-9208-4985-b9a6-d573a4342042 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:39.347248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d103d6-389c-4174-9c2f-2136d9900099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:39.347937 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d103d6-389c-4174-9c2f-2136d9900099 None admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:39.348211 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d103d6-389c-4174-9c2f-2136d9900099 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:39.348476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d103d6-389c-4174-9c2f-2136d9900099 None admin] PATH_INFO: `/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/db581b10d78a4f9eae33fadce0d92b0d/roles/fb5e46782bfc4b09b89697a6b033fecc` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:39.365262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d103d6-389c-4174-9c2f-2136d9900099 None admin] RBAC: Authorizing `identity:create_grant(project_id=12f9875f45a84e92a848b8fc1636d6c6, user_id=db581b10d78a4f9eae33fadce0d92b0d, role_id=fb5e46782bfc4b09b89697a6b033fecc)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:39.367806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d103d6-389c-4174-9c2f-2136d9900099 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:39.383538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 616/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:41:39 2022] PUT /identity/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/db581b10d78a4f9eae33fadce0d92b0d/roles/fb5e46782bfc4b09b89697a6b033fecc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:41:41.388244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29112ea-9557-459c-affa-1a9930c3ceab None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:41.388782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29112ea-9557-459c-affa-1a9930c3ceab None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:41.388913 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29112ea-9557-459c-affa-1a9930c3ceab None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:41.390352 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 617/1233] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:41:41.396717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98021b9f-f29a-424a-9df1-ec1a44449e56 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:41.397232 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98021b9f-f29a-424a-9df1-ec1a44449e56 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:41.397355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98021b9f-f29a-424a-9df1-ec1a44449e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:41.417892 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-98021b9f-f29a-424a-9df1-ec1a44449e56 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:41:41.419241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-98021b9f-f29a-424a-9df1-ec1a44449e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:41:41.490403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 617/1234] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:41:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:41:41.781597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29112ea-9557-459c-affa-1a9930c3ceab None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:41.807170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:41.885545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29112ea-9557-459c-affa-1a9930c3ceab None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:41.922967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffa4a6-8cb6-4f6f-9cc5-962ef116ae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:41.923609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffa4a6-8cb6-4f6f-9cc5-962ef116ae4d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:41.923923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffa4a6-8cb6-4f6f-9cc5-962ef116ae4d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:41.924197 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffa4a6-8cb6-4f6f-9cc5-962ef116ae4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:41.937886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffa4a6-8cb6-4f6f-9cc5-962ef116ae4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-c6ffa4a6-8cb6-4f6f-9cc5-962ef116ae4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:41:41.943153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:41.944315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:41.945518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 618/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Aug 18 14:41:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:41.951339 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98021b9f-f29a-424a-9df1-ec1a44449e56 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:42.000234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:42.065715 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98021b9f-f29a-424a-9df1-ec1a44449e56 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:42.105805 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eef5e1-e4e3-4658-8a89-ee75ae14e9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:42.106464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eef5e1-e4e3-4658-8a89-ee75ae14e9b4 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:42.106741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eef5e1-e4e3-4658-8a89-ee75ae14e9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:42.107012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eef5e1-e4e3-4658-8a89-ee75ae14e9b4 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:42.107778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eef5e1-e4e3-4658-8a89-ee75ae14e9b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:42.108079 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eef5e1-e4e3-4658-8a89-ee75ae14e9b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:42.109730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eef5e1-e4e3-4658-8a89-ee75ae14e9b4 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:42.118990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 618/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Aug 18 14:41:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:42.125120 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffa4a6-8cb6-4f6f-9cc5-962ef116ae4d None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:42.219130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:42.236067 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffa4a6-8cb6-4f6f-9cc5-962ef116ae4d None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:42.272998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b639c42-51d4-43af-a93d-c300cb3289ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:42.273708 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b639c42-51d4-43af-a93d-c300cb3289ae None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:42.274035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b639c42-51d4-43af-a93d-c300cb3289ae None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:42.274314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b639c42-51d4-43af-a93d-c300cb3289ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:42.285377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b639c42-51d4-43af-a93d-c300cb3289ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:42.286825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b639c42-51d4-43af-a93d-c300cb3289ae None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-0b639c42-51d4-43af-a93d-c300cb3289ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:42.302989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:42.304230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:42.305220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:42.305220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:42.305220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:42.305220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 619/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:42.310910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eef5e1-e4e3-4658-8a89-ee75ae14e9b4 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:42.427910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eef5e1-e4e3-4658-8a89-ee75ae14e9b4 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:42.437139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:42.464650 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a656ae-29fe-438c-a5f2-d5ba677e9968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:42.465359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a656ae-29fe-438c-a5f2-d5ba677e9968 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:42.466573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a656ae-29fe-438c-a5f2-d5ba677e9968 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:42.466573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a656ae-29fe-438c-a5f2-d5ba677e9968 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:42.467106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a656ae-29fe-438c-a5f2-d5ba677e9968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:42.467393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a656ae-29fe-438c-a5f2-d5ba677e9968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:42.468769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a656ae-29fe-438c-a5f2-d5ba677e9968 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:42.481194 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 619/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:42.487232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b639c42-51d4-43af-a93d-c300cb3289ae None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:42.595397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b639c42-51d4-43af-a93d-c300cb3289ae None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:42.633880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97efdd-8a77-485d-ab22-ed5b63acb7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:42.634511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97efdd-8a77-485d-ab22-ed5b63acb7d2 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:42.634791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97efdd-8a77-485d-ab22-ed5b63acb7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:42.635064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97efdd-8a77-485d-ab22-ed5b63acb7d2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:42.638267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:42.643693 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97efdd-8a77-485d-ab22-ed5b63acb7d2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:42.646846 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97efdd-8a77-485d-ab22-ed5b63acb7d2 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-ce97efdd-8a77-485d-ab22-ed5b63acb7d2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:41:42.659479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:42.661830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:42.663074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:42.663074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:42.663074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:41:42.663074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:41:42.663074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:41:42.663074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:41:42.663074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:41:42.663074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:41:42.663074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 14:41:42.663074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:42.663074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 620/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:41:42.667492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a656ae-29fe-438c-a5f2-d5ba677e9968 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:42.777616 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a656ae-29fe-438c-a5f2-d5ba677e9968 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:42.816541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3da3a0-2aca-48c7-a4ce-5151c36e7db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:42.817202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3da3a0-2aca-48c7-a4ce-5151c36e7db7 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:42.817518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3da3a0-2aca-48c7-a4ce-5151c36e7db7 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:42.817757 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3da3a0-2aca-48c7-a4ce-5151c36e7db7 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:42.818494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3da3a0-2aca-48c7-a4ce-5151c36e7db7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:42.818800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3da3a0-2aca-48c7-a4ce-5151c36e7db7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:42.820600 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3da3a0-2aca-48c7-a4ce-5151c36e7db7 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:42.840220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 620/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Thu Aug 18 14:41:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:42.847047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97efdd-8a77-485d-ab22-ed5b63acb7d2 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:42.967825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97efdd-8a77-485d-ab22-ed5b63acb7d2 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:43.017613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745c776-d07d-497d-b79e-25030ec515a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:43.018219 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745c776-d07d-497d-b79e-25030ec515a7 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:43.018496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745c776-d07d-497d-b79e-25030ec515a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:43.018769 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745c776-d07d-497d-b79e-25030ec515a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:43.091024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:43.098617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745c776-d07d-497d-b79e-25030ec515a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:43.100120 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745c776-d07d-497d-b79e-25030ec515a7 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:43.203503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 621/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:41:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 14:41:43.209955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3da3a0-2aca-48c7-a4ce-5151c36e7db7 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:43.237835 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:43.317378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3da3a0-2aca-48c7-a4ce-5151c36e7db7 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:43.351381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b60771-bfad-483c-9da2-21bd7ad6abc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:43.351972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b60771-bfad-483c-9da2-21bd7ad6abc3 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:43.352245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b60771-bfad-483c-9da2-21bd7ad6abc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:43.352516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b60771-bfad-483c-9da2-21bd7ad6abc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:43.363717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b60771-bfad-483c-9da2-21bd7ad6abc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:43.365154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b60771-bfad-483c-9da2-21bd7ad6abc3 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-72b60771-bfad-483c-9da2-21bd7ad6abc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:43.380271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:43.381539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:43.382443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:43.382443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:43.382443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:43.382443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 621/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:43.388231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745c776-d07d-497d-b79e-25030ec515a7 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:43.495074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:43.501608 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745c776-d07d-497d-b79e-25030ec515a7 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:43.537182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b6640-1ed7-4896-be62-af7b1bf4b15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:43.537780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b6640-1ed7-4896-be62-af7b1bf4b15f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:43.538054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b6640-1ed7-4896-be62-af7b1bf4b15f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:43.538320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b6640-1ed7-4896-be62-af7b1bf4b15f None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:43.538951 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b6640-1ed7-4896-be62-af7b1bf4b15f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:43.539248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b6640-1ed7-4896-be62-af7b1bf4b15f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:43.540627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b6640-1ed7-4896-be62-af7b1bf4b15f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:43.551733 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 622/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:43.557471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b60771-bfad-483c-9da2-21bd7ad6abc3 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:43.643747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:43.668770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b60771-bfad-483c-9da2-21bd7ad6abc3 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:43.704341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1459a0-d9fd-4360-8603-22bb292b6489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:43.704969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1459a0-d9fd-4360-8603-22bb292b6489 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:43.705242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1459a0-d9fd-4360-8603-22bb292b6489 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:43.705507 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1459a0-d9fd-4360-8603-22bb292b6489 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:43.712653 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1459a0-d9fd-4360-8603-22bb292b6489 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:43.714310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1459a0-d9fd-4360-8603-22bb292b6489 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-ca1459a0-d9fd-4360-8603-22bb292b6489 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:41:43.730723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:43.732467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:43.733960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:41:43.733960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:41:43.733960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:41:43.733960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:41:43.733960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:41:43.733960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:41:43.733960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:41:43.733960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:43.733960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 622/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:41:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:43.738454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b6640-1ed7-4896-be62-af7b1bf4b15f None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:43.846672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b6640-1ed7-4896-be62-af7b1bf4b15f None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:43.882405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939aa17-bf85-4b0b-ac9d-462ac589988f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:43.883136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939aa17-bf85-4b0b-ac9d-462ac589988f None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:43.883403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939aa17-bf85-4b0b-ac9d-462ac589988f None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:43.883749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939aa17-bf85-4b0b-ac9d-462ac589988f None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:43.884453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e939aa17-bf85-4b0b-ac9d-462ac589988f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:43.884806 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e939aa17-bf85-4b0b-ac9d-462ac589988f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:43.886408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e939aa17-bf85-4b0b-ac9d-462ac589988f None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:43.902162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 623/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:41:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:41:43.908960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1459a0-d9fd-4360-8603-22bb292b6489 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:44.017797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1459a0-d9fd-4360-8603-22bb292b6489 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:44.052167 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba33836-95cf-452d-afc5-a71a16f1d4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:44.052803 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba33836-95cf-452d-afc5-a71a16f1d4fb None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:44.053081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba33836-95cf-452d-afc5-a71a16f1d4fb None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:44.053351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba33836-95cf-452d-afc5-a71a16f1d4fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:44.054068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba33836-95cf-452d-afc5-a71a16f1d4fb None admin] RBAC: Adding query filter params (role.id=fb5e46782bfc4b09b89697a6b033fecc, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=db581b10d78a4f9eae33fadce0d92b0d) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:44.054371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba33836-95cf-452d-afc5-a71a16f1d4fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:44.055823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba33836-95cf-452d-afc5-a71a16f1d4fb None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:44.061685 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 623/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:41:43 2022] GET /identity/v3/role_assignments?user.id=db581b10d78a4f9eae33fadce0d92b0d&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=fb5e46782bfc4b09b89697a6b033fecc => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:46.154644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a8c3-54f6-4a01-9584-c70840b83fde None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:46.155150 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a8c3-54f6-4a01-9584-c70840b83fde None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:46.155150 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a8c3-54f6-4a01-9584-c70840b83fde None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:46.156435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 624/1247] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:41:46.161705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf8cb8-ea64-4e99-b429-becc03ddb36e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:46.162159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf8cb8-ea64-4e99-b429-becc03ddb36e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:46.162258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf8cb8-ea64-4e99-b429-becc03ddb36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:46.169229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:46.185378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-f4cf8cb8-ea64-4e99-b429-becc03ddb36e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:41:46.186792 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f4cf8cb8-ea64-4e99-b429-becc03ddb36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:41:46.259248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 624/1248] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:41:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:41:46.525673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975a8c3-54f6-4a01-9584-c70840b83fde None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:46.545547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:46.641134 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975a8c3-54f6-4a01-9584-c70840b83fde None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:46.680428 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe4f57-0676-4aa5-ba56-15226614b069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:46.681684 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe4f57-0676-4aa5-ba56-15226614b069 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:46.682081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe4f57-0676-4aa5-ba56-15226614b069 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:46.682388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe4f57-0676-4aa5-ba56-15226614b069 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:46.696783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe4f57-0676-4aa5-ba56-15226614b069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-54fe4f57-0676-4aa5-ba56-15226614b069 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:41:46.703453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:46.704782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:46.705713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 625/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:41:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:46.710957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf8cb8-ea64-4e99-b429-becc03ddb36e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:46.821257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf8cb8-ea64-4e99-b429-becc03ddb36e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:46.855361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:46.861019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01788d-293f-47f5-b747-6156e09c37e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:46.861744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01788d-293f-47f5-b747-6156e09c37e2 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:46.862066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01788d-293f-47f5-b747-6156e09c37e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:46.862358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01788d-293f-47f5-b747-6156e09c37e2 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:46.863091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01788d-293f-47f5-b747-6156e09c37e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:46.863391 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01788d-293f-47f5-b747-6156e09c37e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:46.864947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01788d-293f-47f5-b747-6156e09c37e2 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:46.874445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 625/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:41:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:46.880652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe4f57-0676-4aa5-ba56-15226614b069 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:46.966886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:46.994816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe4f57-0676-4aa5-ba56-15226614b069 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:47.033511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1099b-ca81-4687-8952-6b2b696bb288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:47.034311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1099b-ca81-4687-8952-6b2b696bb288 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:47.034638 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1099b-ca81-4687-8952-6b2b696bb288 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:47.034968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1099b-ca81-4687-8952-6b2b696bb288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:47.047431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1099b-ca81-4687-8952-6b2b696bb288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:47.049083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1099b-ca81-4687-8952-6b2b696bb288 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-20b1099b-ca81-4687-8952-6b2b696bb288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:47.066324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:47.067341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:47.068723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:47.068723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:47.068723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:47.068723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 626/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:47.074342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01788d-293f-47f5-b747-6156e09c37e2 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:47.181218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01788d-293f-47f5-b747-6156e09c37e2 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:47.224698 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581ce08-01f5-4319-9dbb-2fb21b25553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:47.225531 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581ce08-01f5-4319-9dbb-2fb21b25553f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:47.225899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581ce08-01f5-4319-9dbb-2fb21b25553f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:47.226376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581ce08-01f5-4319-9dbb-2fb21b25553f None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:47.228127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581ce08-01f5-4319-9dbb-2fb21b25553f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:47.228513 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581ce08-01f5-4319-9dbb-2fb21b25553f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:47.229900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581ce08-01f5-4319-9dbb-2fb21b25553f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:47.243211 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 626/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:47.249803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1099b-ca81-4687-8952-6b2b696bb288 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:47.356744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1099b-ca81-4687-8952-6b2b696bb288 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:47.384891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:47.394551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d919a-aa3e-4965-a6b8-f2d9287d9241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:47.395178 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d919a-aa3e-4965-a6b8-f2d9287d9241 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:47.395487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d919a-aa3e-4965-a6b8-f2d9287d9241 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:47.395880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d919a-aa3e-4965-a6b8-f2d9287d9241 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:47.403982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d919a-aa3e-4965-a6b8-f2d9287d9241 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:47.405463 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d919a-aa3e-4965-a6b8-f2d9287d9241 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-2e4d919a-aa3e-4965-a6b8-f2d9287d9241 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:41:47.419210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:47.420878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:47.422314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:47.422314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:47.422314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:41:47.422314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:41:47.422314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:41:47.422314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:41:47.422314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:41:47.422314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:41:47.422314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 14:41:47.422314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:47.422314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 627/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:47.427827 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581ce08-01f5-4319-9dbb-2fb21b25553f None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:47.494510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:47.545258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581ce08-01f5-4319-9dbb-2fb21b25553f None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:47.579513 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1613daf-6084-4f5c-b8b3-7ed277585743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:47.580146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1613daf-6084-4f5c-b8b3-7ed277585743 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:47.580421 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1613daf-6084-4f5c-b8b3-7ed277585743 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:47.580688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1613daf-6084-4f5c-b8b3-7ed277585743 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:47.581401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1613daf-6084-4f5c-b8b3-7ed277585743 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:47.581700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1613daf-6084-4f5c-b8b3-7ed277585743 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:47.583114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1613daf-6084-4f5c-b8b3-7ed277585743 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:47.602906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 627/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Thu Aug 18 14:41:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:47.608958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d919a-aa3e-4965-a6b8-f2d9287d9241 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:47.711963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d919a-aa3e-4965-a6b8-f2d9287d9241 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:47.754689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136dfa5-d0df-4a27-8a76-4eab64bebe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:47.755324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136dfa5-d0df-4a27-8a76-4eab64bebe0b None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:47.755604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136dfa5-d0df-4a27-8a76-4eab64bebe0b None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:47.755904 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136dfa5-d0df-4a27-8a76-4eab64bebe0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:47.828188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:47.843305 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136dfa5-d0df-4a27-8a76-4eab64bebe0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:47.845162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136dfa5-d0df-4a27-8a76-4eab64bebe0b None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:47.956148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 628/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:41:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:41:47.961749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1613daf-6084-4f5c-b8b3-7ed277585743 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:48.080811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1613daf-6084-4f5c-b8b3-7ed277585743 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:48.094980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:48.117105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ffe62-c11f-426e-894f-e4d36e458b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:48.117733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ffe62-c11f-426e-894f-e4d36e458b42 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:48.118011 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ffe62-c11f-426e-894f-e4d36e458b42 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:48.118278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ffe62-c11f-426e-894f-e4d36e458b42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:48.129600 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ffe62-c11f-426e-894f-e4d36e458b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:48.131214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ffe62-c11f-426e-894f-e4d36e458b42 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-379ffe62-c11f-426e-894f-e4d36e458b42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:48.148097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:48.149186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:48.150109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:48.150109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:48.150109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:48.150109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 628/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 14:41:48.155648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136dfa5-d0df-4a27-8a76-4eab64bebe0b None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:48.250900 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:48.264794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136dfa5-d0df-4a27-8a76-4eab64bebe0b None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:48.302524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbf985-d5cf-4a6a-8c35-482b496a1cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:48.303126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbf985-d5cf-4a6a-8c35-482b496a1cb3 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:48.303498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbf985-d5cf-4a6a-8c35-482b496a1cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:48.303671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbf985-d5cf-4a6a-8c35-482b496a1cb3 None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:48.304325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbf985-d5cf-4a6a-8c35-482b496a1cb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:48.304620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbf985-d5cf-4a6a-8c35-482b496a1cb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:48.305970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbf985-d5cf-4a6a-8c35-482b496a1cb3 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:48.318172 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 629/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:48.324591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ffe62-c11f-426e-894f-e4d36e458b42 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:48.439195 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ffe62-c11f-426e-894f-e4d36e458b42 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:48.474799 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12dc16-cda3-4203-860b-f1a0a842f0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:48.475617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12dc16-cda3-4203-860b-f1a0a842f0a1 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:48.475923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12dc16-cda3-4203-860b-f1a0a842f0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:48.476195 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12dc16-cda3-4203-860b-f1a0a842f0a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:48.484559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12dc16-cda3-4203-860b-f1a0a842f0a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:48.486199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12dc16-cda3-4203-860b-f1a0a842f0a1 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-7c12dc16-cda3-4203-860b-f1a0a842f0a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:41:48.500179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:48.501288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:48.502598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:41:48.502598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:41:48.502598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:41:48.502598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:41:48.502598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:41:48.502598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:41:48.502598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:41:48.502598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:48.502598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 629/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:41:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:48.508032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbf985-d5cf-4a6a-8c35-482b496a1cb3 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:48.616787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbf985-d5cf-4a6a-8c35-482b496a1cb3 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:48.656278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc01eb-82fb-476e-a9d5-31e320923b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:48.656957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc01eb-82fb-476e-a9d5-31e320923b7e None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:48.657808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc01eb-82fb-476e-a9d5-31e320923b7e None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:48.657808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc01eb-82fb-476e-a9d5-31e320923b7e None admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:48.658187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc01eb-82fb-476e-a9d5-31e320923b7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:48.658494 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc01eb-82fb-476e-a9d5-31e320923b7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:48.660032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc01eb-82fb-476e-a9d5-31e320923b7e None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:48.673498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 630/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:41:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:48.679868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12dc16-cda3-4203-860b-f1a0a842f0a1 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:48.709252 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:48.788248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12dc16-cda3-4203-860b-f1a0a842f0a1 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:48.823647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde20fe9-7390-4181-9a19-286cd9f30b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:48.824291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20fe9-7390-4181-9a19-286cd9f30b1f None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:48.824566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20fe9-7390-4181-9a19-286cd9f30b1f None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:48.825325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20fe9-7390-4181-9a19-286cd9f30b1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:48.826054 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20fe9-7390-4181-9a19-286cd9f30b1f None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=db581b10d78a4f9eae33fadce0d92b0d) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:48.826356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20fe9-7390-4181-9a19-286cd9f30b1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:48.827873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20fe9-7390-4181-9a19-286cd9f30b1f None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:48.834593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 630/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:41:48 2022] GET /identity/v3/role_assignments?user.id=db581b10d78a4f9eae33fadce0d92b0d&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=9a439cf8eef34ada85428b378878e63e => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:50.739741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ec02a-c134-4320-ad3d-f19b7a2d9e8a None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:50.740295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ec02a-c134-4320-ad3d-f19b7a2d9e8a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:50.740295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ec02a-c134-4320-ad3d-f19b7a2d9e8a None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:50.741623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 631/1261] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:41:50.747380 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc356ea-0206-4985-bf1a-a808e3c20681 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:50.747831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc356ea-0206-4985-bf1a-a808e3c20681 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:50.747980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc356ea-0206-4985-bf1a-a808e3c20681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:50.765791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-7fc356ea-0206-4985-bf1a-a808e3c20681 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:41:50.766961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7fc356ea-0206-4985-bf1a-a808e3c20681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:41:50.829494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 631/1262] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:41:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:41:51.095528 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ec02a-c134-4320-ad3d-f19b7a2d9e8a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:51.106258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:51.194070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ec02a-c134-4320-ad3d-f19b7a2d9e8a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:51.226407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5ded3-c36a-4de6-9ff9-17fcae60e3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:51.227013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5ded3-c36a-4de6-9ff9-17fcae60e3af None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:51.227297 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5ded3-c36a-4de6-9ff9-17fcae60e3af None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:51.227572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5ded3-c36a-4de6-9ff9-17fcae60e3af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:51.237451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5ded3-c36a-4de6-9ff9-17fcae60e3af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-14e5ded3-c36a-4de6-9ff9-17fcae60e3af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:41:51.242141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:51.243231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:51.244303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 632/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:41:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:51.249935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc356ea-0206-4985-bf1a-a808e3c20681 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:51.268572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:51.347086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc356ea-0206-4985-bf1a-a808e3c20681 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:51.383005 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e707d3-9062-49d7-9005-c53b4f0995e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:51.383731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e707d3-9062-49d7-9005-c53b4f0995e5 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:51.384040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e707d3-9062-49d7-9005-c53b4f0995e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:51.384290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e707d3-9062-49d7-9005-c53b4f0995e5 None admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:51.384949 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e707d3-9062-49d7-9005-c53b4f0995e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:51.385248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e707d3-9062-49d7-9005-c53b4f0995e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:51.386623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e707d3-9062-49d7-9005-c53b4f0995e5 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:51.394984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 632/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:41:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:51.400965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5ded3-c36a-4de6-9ff9-17fcae60e3af None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:51.473813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:51.505653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5ded3-c36a-4de6-9ff9-17fcae60e3af None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:51.538488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22965606-5a1e-4e2c-bbf9-26cd2f0b2ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:51.539105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22965606-5a1e-4e2c-bbf9-26cd2f0b2ee1 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:51.539384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22965606-5a1e-4e2c-bbf9-26cd2f0b2ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:51.539657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22965606-5a1e-4e2c-bbf9-26cd2f0b2ee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:51.549950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22965606-5a1e-4e2c-bbf9-26cd2f0b2ee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:51.551388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22965606-5a1e-4e2c-bbf9-26cd2f0b2ee1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-22965606-5a1e-4e2c-bbf9-26cd2f0b2ee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:51.565394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:51.566419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:51.567548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:51.567548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:51.567548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:51.567548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 633/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:51.572658 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e707d3-9062-49d7-9005-c53b4f0995e5 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:51.650223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:51.669039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e707d3-9062-49d7-9005-c53b4f0995e5 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:51.700746 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad9b18-2887-44e4-959a-039e1081d835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:51.701333 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad9b18-2887-44e4-959a-039e1081d835 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:51.701605 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad9b18-2887-44e4-959a-039e1081d835 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:51.701870 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad9b18-2887-44e4-959a-039e1081d835 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:51.702496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad9b18-2887-44e4-959a-039e1081d835 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:51.702790 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad9b18-2887-44e4-959a-039e1081d835 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:51.704180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad9b18-2887-44e4-959a-039e1081d835 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:51.716002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 633/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:51.721419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22965606-5a1e-4e2c-bbf9-26cd2f0b2ee1 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:51.817530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22965606-5a1e-4e2c-bbf9-26cd2f0b2ee1 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:51.836284 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:51.850162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6c405-6331-437a-89b6-ac60ff47a7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:51.850768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6c405-6331-437a-89b6-ac60ff47a7c0 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:51.851039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6c405-6331-437a-89b6-ac60ff47a7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:51.851302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6c405-6331-437a-89b6-ac60ff47a7c0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:51.858322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6c405-6331-437a-89b6-ac60ff47a7c0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:51.859775 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6c405-6331-437a-89b6-ac60ff47a7c0 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-12c6c405-6331-437a-89b6-ac60ff47a7c0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:41:51.870273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:51.871361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:51.873180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:51.873180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:51.873180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:41:51.873180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:41:51.873180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:41:51.873180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:41:51.873180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:41:51.873180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:41:51.873180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 14:41:51.873180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:51.873180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 634/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:51.877768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad9b18-2887-44e4-959a-039e1081d835 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:51.978941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad9b18-2887-44e4-959a-039e1081d835 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:52.011918 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299969f-6329-421f-b14d-059374f23c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:52.013048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299969f-6329-421f-b14d-059374f23c31 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:52.013324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299969f-6329-421f-b14d-059374f23c31 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:52.013595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299969f-6329-421f-b14d-059374f23c31 None admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:52.014309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299969f-6329-421f-b14d-059374f23c31 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:52.014613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299969f-6329-421f-b14d-059374f23c31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:52.016061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299969f-6329-421f-b14d-059374f23c31 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:52.024485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:52.033774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 634/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Thu Aug 18 14:41:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:52.039461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6c405-6331-437a-89b6-ac60ff47a7c0 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:52.136147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6c405-6331-437a-89b6-ac60ff47a7c0 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:52.168999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9e45c-413f-4be2-822b-6cfe403b5381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:52.169602 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9e45c-413f-4be2-822b-6cfe403b5381 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:52.169879 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9e45c-413f-4be2-822b-6cfe403b5381 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:52.170148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9e45c-413f-4be2-822b-6cfe403b5381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:52.180146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9e45c-413f-4be2-822b-6cfe403b5381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:52.181579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9e45c-413f-4be2-822b-6cfe403b5381 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-fec9e45c-413f-4be2-822b-6cfe403b5381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:52.195000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:52.196158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:52.197283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:52.197283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:52.197283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:52.197283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 635/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:52.202487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299969f-6329-421f-b14d-059374f23c31 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:52.300002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299969f-6329-421f-b14d-059374f23c31 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:52.332584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ff605-09cb-4e1b-ac09-36a166a01b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:52.333174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ff605-09cb-4e1b-ac09-36a166a01b01 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:52.333453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ff605-09cb-4e1b-ac09-36a166a01b01 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:52.333723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ff605-09cb-4e1b-ac09-36a166a01b01 None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:52.334354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ff605-09cb-4e1b-ac09-36a166a01b01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:52.334654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ff605-09cb-4e1b-ac09-36a166a01b01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:52.336042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ff605-09cb-4e1b-ac09-36a166a01b01 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:52.346897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 635/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:52.352533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9e45c-413f-4be2-822b-6cfe403b5381 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:52.385488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:52.456897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9e45c-413f-4be2-822b-6cfe403b5381 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:52.488804 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963001e-d12d-422c-af1b-605710784914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:52.489422 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963001e-d12d-422c-af1b-605710784914 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:52.489706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963001e-d12d-422c-af1b-605710784914 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:52.489977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963001e-d12d-422c-af1b-605710784914 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:52.496309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963001e-d12d-422c-af1b-605710784914 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:52.497787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963001e-d12d-422c-af1b-605710784914 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-9963001e-d12d-422c-af1b-605710784914 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:41:52.508876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:52.509790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:52.510950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:41:52.510950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:41:52.510950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:41:52.510950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:41:52.510950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:41:52.510950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:41:52.510950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:41:52.510950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:52.510950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 636/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:41:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:52.516378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ff605-09cb-4e1b-ac09-36a166a01b01 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:52.572927 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:52.614962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ff605-09cb-4e1b-ac09-36a166a01b01 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:52.648495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf948c-20b8-4b41-b80a-9c429e5c19ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:52.649090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf948c-20b8-4b41-b80a-9c429e5c19ea None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:52.649370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf948c-20b8-4b41-b80a-9c429e5c19ea None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:52.649641 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf948c-20b8-4b41-b80a-9c429e5c19ea None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:52.650325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf948c-20b8-4b41-b80a-9c429e5c19ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:52.650631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf948c-20b8-4b41-b80a-9c429e5c19ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:52.652089 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf948c-20b8-4b41-b80a-9c429e5c19ea None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:52.663566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 636/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:41:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:52.669647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963001e-d12d-422c-af1b-605710784914 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:52.759315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:52.768593 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963001e-d12d-422c-af1b-605710784914 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:52.799566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa9b79-8627-4942-a7d7-fe62da956095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:52.800229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa9b79-8627-4942-a7d7-fe62da956095 None admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:52.800503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa9b79-8627-4942-a7d7-fe62da956095 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:52.800769 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa9b79-8627-4942-a7d7-fe62da956095 None admin] PATH_INFO: `/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/db581b10d78a4f9eae33fadce0d92b0d/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:52.802638 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa9b79-8627-4942-a7d7-fe62da956095 None admin] RBAC: Authorizing `identity:create_grant(project_id=12f9875f45a84e92a848b8fc1636d6c6, user_id=db581b10d78a4f9eae33fadce0d92b0d, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:52.804171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa9b79-8627-4942-a7d7-fe62da956095 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:52.816610 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 637/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Aug 18 14:41:52 2022] PUT /identity/v3/projects/12f9875f45a84e92a848b8fc1636d6c6/users/db581b10d78a4f9eae33fadce0d92b0d/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:41:54.677482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173d66c-0ece-4d61-8568-18d234f59e69 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:54.678051 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173d66c-0ece-4d61-8568-18d234f59e69 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:54.678051 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173d66c-0ece-4d61-8568-18d234f59e69 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:54.679332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 637/1274] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:41:54.684330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25b0a3-3f5b-4371-91fc-8a22abd4966b None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:54.684752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25b0a3-3f5b-4371-91fc-8a22abd4966b None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:54.684870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25b0a3-3f5b-4371-91fc-8a22abd4966b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:54.702865 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-ce25b0a3-3f5b-4371-91fc-8a22abd4966b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:41:54.704100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ce25b0a3-3f5b-4371-91fc-8a22abd4966b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:41:54.767536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 638/1275] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:41:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:41:55.033017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173d66c-0ece-4d61-8568-18d234f59e69 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:55.131062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173d66c-0ece-4d61-8568-18d234f59e69 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:55.147420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:55.164105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7715e-7bf7-4cbb-9289-052208a45481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:55.164721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7715e-7bf7-4cbb-9289-052208a45481 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:55.164997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7715e-7bf7-4cbb-9289-052208a45481 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:55.165267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7715e-7bf7-4cbb-9289-052208a45481 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:55.174965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7715e-7bf7-4cbb-9289-052208a45481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-65d7715e-7bf7-4cbb-9289-052208a45481 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 14:41:55.179593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:55.180741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:55.181784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 638/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Aug 18 14:41:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:55.187192 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25b0a3-3f5b-4371-91fc-8a22abd4966b None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:55.263148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:55.280654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25b0a3-3f5b-4371-91fc-8a22abd4966b None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:55.312435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1f9a1-9426-44e7-974e-44a5677e1e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:55.312959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1f9a1-9426-44e7-974e-44a5677e1e97 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:55.313232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1f9a1-9426-44e7-974e-44a5677e1e97 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:55.313498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1f9a1-9426-44e7-974e-44a5677e1e97 None admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:55.314144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1f9a1-9426-44e7-974e-44a5677e1e97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:55.314440 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1f9a1-9426-44e7-974e-44a5677e1e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:55.315814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1f9a1-9426-44e7-974e-44a5677e1e97 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:55.323312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 639/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Aug 18 14:41:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:55.329521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7715e-7bf7-4cbb-9289-052208a45481 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:55.430617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7715e-7bf7-4cbb-9289-052208a45481 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:55.462995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb51d6f-6579-4426-86dd-81c30477fa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:55.463602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb51d6f-6579-4426-86dd-81c30477fa3d None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:55.463906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb51d6f-6579-4426-86dd-81c30477fa3d None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:55.464174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb51d6f-6579-4426-86dd-81c30477fa3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:55.473710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb51d6f-6579-4426-86dd-81c30477fa3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:55.475133 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb51d6f-6579-4426-86dd-81c30477fa3d None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-ecb51d6f-6579-4426-86dd-81c30477fa3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:55.489443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:55.490561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:55.491703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:55.491703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:55.491703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:55.491703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 639/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:55.497112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1f9a1-9426-44e7-974e-44a5677e1e97 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:55.591726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1f9a1-9426-44e7-974e-44a5677e1e97 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:55.622904 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28134a9a-9cee-4543-9b53-0eb47408577a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:55.623482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28134a9a-9cee-4543-9b53-0eb47408577a None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:55.623788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28134a9a-9cee-4543-9b53-0eb47408577a None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:55.624058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28134a9a-9cee-4543-9b53-0eb47408577a None admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:55.624681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28134a9a-9cee-4543-9b53-0eb47408577a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:55.624978 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28134a9a-9cee-4543-9b53-0eb47408577a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:55.626329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28134a9a-9cee-4543-9b53-0eb47408577a None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:55.637579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 640/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:55.643130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb51d6f-6579-4426-86dd-81c30477fa3d None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:55.674337 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:55.741224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb51d6f-6579-4426-86dd-81c30477fa3d None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:55.772457 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ce0cb-713b-4f40-b05c-1a02e14e3d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:55.773052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ce0cb-713b-4f40-b05c-1a02e14e3d2a None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:55.773325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ce0cb-713b-4f40-b05c-1a02e14e3d2a None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:55.773595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ce0cb-713b-4f40-b05c-1a02e14e3d2a None admin] PATH_INFO: `/v3/users/placement` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:55.781091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ce0cb-713b-4f40-b05c-1a02e14e3d2a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:55.782503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ce0cb-713b-4f40-b05c-1a02e14e3d2a None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-461ce0cb-713b-4f40-b05c-1a02e14e3d2a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 14:41:55.793066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:55.794201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:55.795557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:55.795557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:55.795557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 14:41:55.795557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 14:41:55.795557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 14:41:55.795557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 14:41:55.795557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 14:41:55.795557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 14:41:55.795557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 14:41:55.795557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:55.795557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 640/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:55.799845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28134a9a-9cee-4543-9b53-0eb47408577a None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:55.803511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:55.894188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28134a9a-9cee-4543-9b53-0eb47408577a None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:55.927566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36059519-b05b-435a-bf30-fb588d87b078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:55.928210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36059519-b05b-435a-bf30-fb588d87b078 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:55.928485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36059519-b05b-435a-bf30-fb588d87b078 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:55.928753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36059519-b05b-435a-bf30-fb588d87b078 None admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:55.929464 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36059519-b05b-435a-bf30-fb588d87b078 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:55.929765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36059519-b05b-435a-bf30-fb588d87b078 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:55.931199 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36059519-b05b-435a-bf30-fb588d87b078 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:55.947342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 641/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Thu Aug 18 14:41:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:41:55.953501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ce0cb-713b-4f40-b05c-1a02e14e3d2a None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:56.040048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:56.050215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ce0cb-713b-4f40-b05c-1a02e14e3d2a None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:56.082424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a3ca1-ef79-47b2-8ff6-ccdb04e0db0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:56.083011 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a3ca1-ef79-47b2-8ff6-ccdb04e0db0b None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:56.083290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a3ca1-ef79-47b2-8ff6-ccdb04e0db0b None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:56.083560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a3ca1-ef79-47b2-8ff6-ccdb04e0db0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:56.152397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a3ca1-ef79-47b2-8ff6-ccdb04e0db0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:56.153808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a3ca1-ef79-47b2-8ff6-ccdb04e0db0b None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:56.237730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:56.248416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 641/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Aug 18 14:41:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 14:41:56.254566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36059519-b05b-435a-bf30-fb588d87b078 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:56.313660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:56.350189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36059519-b05b-435a-bf30-fb588d87b078 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:56.387050 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b613a56-0b55-4366-a889-63b3eb2b0463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:56.387665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b613a56-0b55-4366-a889-63b3eb2b0463 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:56.387970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b613a56-0b55-4366-a889-63b3eb2b0463 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:56.388238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b613a56-0b55-4366-a889-63b3eb2b0463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:56.401684 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b613a56-0b55-4366-a889-63b3eb2b0463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:56.403694 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b613a56-0b55-4366-a889-63b3eb2b0463 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-1b613a56-0b55-4366-a889-63b3eb2b0463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 14:41:56.420085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:56.421329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 14:41:56.422557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 14:41:56.422557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 14:41:56.422557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:56.422557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 642/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:56.426747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a3ca1-ef79-47b2-8ff6-ccdb04e0db0b None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:56.521717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a3ca1-ef79-47b2-8ff6-ccdb04e0db0b None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:56.555179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa16e0-9a73-4be3-a13f-44dcb36fbb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:56.555806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa16e0-9a73-4be3-a13f-44dcb36fbb5e None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:56.556084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa16e0-9a73-4be3-a13f-44dcb36fbb5e None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:56.556357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa16e0-9a73-4be3-a13f-44dcb36fbb5e None admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:56.556982 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa16e0-9a73-4be3-a13f-44dcb36fbb5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:56.557284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa16e0-9a73-4be3-a13f-44dcb36fbb5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:56.558634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa16e0-9a73-4be3-a13f-44dcb36fbb5e None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:56.569425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 642/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Aug 18 14:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:56.574813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b613a56-0b55-4366-a889-63b3eb2b0463 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:56.677457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b613a56-0b55-4366-a889-63b3eb2b0463 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:56.690556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:56.710573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a263d-5835-41f9-af39-0b708e970b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:56.711193 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a263d-5835-41f9-af39-0b708e970b60 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:56.711474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a263d-5835-41f9-af39-0b708e970b60 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:56.711778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a263d-5835-41f9-af39-0b708e970b60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:56.718712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a263d-5835-41f9-af39-0b708e970b60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:56.720203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a263d-5835-41f9-af39-0b708e970b60 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application [None req-3e0a263d-5835-41f9-af39-0b708e970b60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 14:41:56.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:56.732171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:56.733263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 14:41:56.733263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 14:41:56.733263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 14:41:56.733263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 14:41:56.733263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 14:41:56.733263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 14:41:56.733263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 14:41:56.733263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: ERROR keystone.server.flask.application Aug 18 14:41:56.733263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 643/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Aug 18 14:41:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:56.738188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa16e0-9a73-4be3-a13f-44dcb36fbb5e None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:56.788349 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:56.837004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa16e0-9a73-4be3-a13f-44dcb36fbb5e None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:56.869632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b466480-6e52-4461-80fc-05481e63a71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:56.870220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b466480-6e52-4461-80fc-05481e63a71c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:56.870497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b466480-6e52-4461-80fc-05481e63a71c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:56.870771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b466480-6e52-4461-80fc-05481e63a71c None admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:56.871479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b466480-6e52-4461-80fc-05481e63a71c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:56.871813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b466480-6e52-4461-80fc-05481e63a71c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:56.873235 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b466480-6e52-4461-80fc-05481e63a71c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:56.885653 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 643/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Aug 18 14:41:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:41:56.892029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a263d-5835-41f9-af39-0b708e970b60 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:56.989079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a263d-5835-41f9-af39-0b708e970b60 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:57.021065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfecba-ff4f-4b6f-a4ce-2a8a417015d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:57.021653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfecba-ff4f-4b6f-a4ce-2a8a417015d4 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:57.021932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfecba-ff4f-4b6f-a4ce-2a8a417015d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:57.022203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfecba-ff4f-4b6f-a4ce-2a8a417015d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:57.022914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfecba-ff4f-4b6f-a4ce-2a8a417015d4 None admin] RBAC: Adding query filter params (role.id=9a439cf8eef34ada85428b378878e63e, scope.project.id=12f9875f45a84e92a848b8fc1636d6c6, user.id=db581b10d78a4f9eae33fadce0d92b0d) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:57.023218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfecba-ff4f-4b6f-a4ce-2a8a417015d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:57.024671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfecba-ff4f-4b6f-a4ce-2a8a417015d4 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:57.030710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 644/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Aug 18 14:41:56 2022] GET /identity/v3/role_assignments?user.id=db581b10d78a4f9eae33fadce0d92b0d&scope.project.id=12f9875f45a84e92a848b8fc1636d6c6&role.id=9a439cf8eef34ada85428b378878e63e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:58.909008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf3f53-2c10-4048-99d2-3f8fa8e65c56 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:58.909400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf3f53-2c10-4048-99d2-3f8fa8e65c56 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:58.909551 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf3f53-2c10-4048-99d2-3f8fa8e65c56 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:58.910879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 644/1288] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:41:58.916051 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c08d1-62e8-499d-8449-3fe1fbe6ad73 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:58.916452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c08d1-62e8-499d-8449-3fe1fbe6ad73 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:58.916600 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c08d1-62e8-499d-8449-3fe1fbe6ad73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:58.934858 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-8d4c08d1-62e8-499d-8449-3fe1fbe6ad73 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:41:58.936086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8d4c08d1-62e8-499d-8449-3fe1fbe6ad73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:41:58.964515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:59.001152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 645/1289] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:41:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:41:59.265868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf3f53-2c10-4048-99d2-3f8fa8e65c56 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:59.363415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf3f53-2c10-4048-99d2-3f8fa8e65c56 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:59.376953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:41:59.404600 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60f94f-bbee-4719-8295-8fbd6788ab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:59.405227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60f94f-bbee-4719-8295-8fbd6788ab01 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:59.405548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60f94f-bbee-4719-8295-8fbd6788ab01 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:59.405818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60f94f-bbee-4719-8295-8fbd6788ab01 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:59.406397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60f94f-bbee-4719-8295-8fbd6788ab01 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:59.407849 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60f94f-bbee-4719-8295-8fbd6788ab01 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-9f60f94f-bbee-4719-8295-8fbd6788ab01 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:41:59.416552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:41:59.417754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:41:59.418696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 645/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Aug 18 14:41:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:41:59.424845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c08d1-62e8-499d-8449-3fe1fbe6ad73 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:59.520875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c08d1-62e8-499d-8449-3fe1fbe6ad73 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:59.553590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086df19e-f22c-4f55-8a55-defb040cf151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:59.554182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086df19e-f22c-4f55-8a55-defb040cf151 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:59.554458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086df19e-f22c-4f55-8a55-defb040cf151 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:59.554725 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086df19e-f22c-4f55-8a55-defb040cf151 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:59.555344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086df19e-f22c-4f55-8a55-defb040cf151 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:59.555645 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086df19e-f22c-4f55-8a55-defb040cf151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:59.557028 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086df19e-f22c-4f55-8a55-defb040cf151 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:59.561662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 646/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Thu Aug 18 14:41:59 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:41:59.567322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60f94f-bbee-4719-8295-8fbd6788ab01 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:41:59.664086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60f94f-bbee-4719-8295-8fbd6788ab01 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:41:59.696171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7c21e-8eb8-46e8-9d41-2045ca178951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:41:59.696758 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7c21e-8eb8-46e8-9d41-2045ca178951 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:41:59.697037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7c21e-8eb8-46e8-9d41-2045ca178951 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:41:59.697308 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7c21e-8eb8-46e8-9d41-2045ca178951 None admin] PATH_INFO: `/v3/services` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:41:59.697926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7c21e-8eb8-46e8-9d41-2045ca178951 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:41:59.698226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7c21e-8eb8-46e8-9d41-2045ca178951 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:41:59.700129 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7c21e-8eb8-46e8-9d41-2045ca178951 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:41:59.704399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 646/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Thu Aug 18 14:41:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:42:01.563361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213e33c-8387-4b7f-bd36-2c3ddc72ca0c None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:01.563923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213e33c-8387-4b7f-bd36-2c3ddc72ca0c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:01.563923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213e33c-8387-4b7f-bd36-2c3ddc72ca0c None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:01.565246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 647/1293] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:42:01.570315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adece8-fecf-471b-973a-07476c4e50a6 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:01.570692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adece8-fecf-471b-973a-07476c4e50a6 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:01.570873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adece8-fecf-471b-973a-07476c4e50a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:01.589056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-62adece8-fecf-471b-973a-07476c4e50a6 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:42:01.590811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-62adece8-fecf-471b-973a-07476c4e50a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:42:01.619116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:42:01.656755 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 647/1294] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:42:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:42:01.920874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213e33c-8387-4b7f-bd36-2c3ddc72ca0c None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:42:01.950989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:42:02.018697 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213e33c-8387-4b7f-bd36-2c3ddc72ca0c None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:42:02.050668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0fae3-599f-4954-a187-eb3a408cd9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:42:02.051256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0fae3-599f-4954-a187-eb3a408cd9d1 None admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:02.051534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0fae3-599f-4954-a187-eb3a408cd9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:02.051831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0fae3-599f-4954-a187-eb3a408cd9d1 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:02.052465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0fae3-599f-4954-a187-eb3a408cd9d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:42:02.053811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0fae3-599f-4954-a187-eb3a408cd9d1 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:42:02.067834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 648/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Aug 18 14:42:01 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:42:03.933605 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516fb57-8fe3-4142-8980-a7915562788d None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:03.934106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516fb57-8fe3-4142-8980-a7915562788d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:03.934214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516fb57-8fe3-4142-8980-a7915562788d None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:03.935495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 648/1296] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:42:03.940457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a4eb0-8476-4850-949d-c58390e2f9e4 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:03.940906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a4eb0-8476-4850-949d-c58390e2f9e4 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:03.941006 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a4eb0-8476-4850-949d-c58390e2f9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:03.958442 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-9a7a4eb0-8476-4850-949d-c58390e2f9e4 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:42:03.959624 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9a7a4eb0-8476-4850-949d-c58390e2f9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:42:04.029828 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 649/1297] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:42:03 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:42:04.294848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516fb57-8fe3-4142-8980-a7915562788d None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:42:04.401119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516fb57-8fe3-4142-8980-a7915562788d None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:42:04.441336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfbaa2-2ff6-44e2-946f-cadbfcb8efb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:42:04.441969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfbaa2-2ff6-44e2-946f-cadbfcb8efb0 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:04.442248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfbaa2-2ff6-44e2-946f-cadbfcb8efb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:04.442518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfbaa2-2ff6-44e2-946f-cadbfcb8efb0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:04.443101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfbaa2-2ff6-44e2-946f-cadbfcb8efb0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:42:04.444506 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfbaa2-2ff6-44e2-946f-cadbfcb8efb0 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-49bfbaa2-2ff6-44e2-946f-cadbfcb8efb0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:42:04.454062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:42:04.455359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:42:04.456510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 649/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Aug 18 14:42:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:42:04.461358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a4eb0-8476-4850-949d-c58390e2f9e4 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:42:04.478896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:42:04.557152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a4eb0-8476-4850-949d-c58390e2f9e4 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:42:04.589552 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605dc4e-41c2-42f8-8c53-00b91680f587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:42:04.590666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605dc4e-41c2-42f8-8c53-00b91680f587 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:04.590946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605dc4e-41c2-42f8-8c53-00b91680f587 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:04.591220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605dc4e-41c2-42f8-8c53-00b91680f587 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:04.591876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605dc4e-41c2-42f8-8c53-00b91680f587 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:42:04.592182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605dc4e-41c2-42f8-8c53-00b91680f587 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:42:04.593545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605dc4e-41c2-42f8-8c53-00b91680f587 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:42:04.598463 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 650/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Thu Aug 18 14:42:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:42:04.604360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfbaa2-2ff6-44e2-946f-cadbfcb8efb0 None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:42:04.635372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:42:04.700557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfbaa2-2ff6-44e2-946f-cadbfcb8efb0 None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:42:04.733862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0319b4e-f9cb-4638-a54e-608336aa9726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:42:04.734454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0319b4e-f9cb-4638-a54e-608336aa9726 None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:04.734728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0319b4e-f9cb-4638-a54e-608336aa9726 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:04.734994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0319b4e-f9cb-4638-a54e-608336aa9726 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:04.735706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0319b4e-f9cb-4638-a54e-608336aa9726 None admin] RBAC: Adding query filter params (interface=public, service_id=b1b379b6e1944ca7b1086b64737eff35, region_id=RegionOne) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:42:04.736006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0319b4e-f9cb-4638-a54e-608336aa9726 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:42:04.737344 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0319b4e-f9cb-4638-a54e-608336aa9726 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:42:04.742126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 650/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Aug 18 14:42:04 2022] GET /identity/v3/endpoints?service_id=b1b379b6e1944ca7b1086b64737eff35&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:42:04.747824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605dc4e-41c2-42f8-8c53-00b91680f587 None admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:42:04.825522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:42:04.843599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605dc4e-41c2-42f8-8c53-00b91680f587 None admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:42:04.875205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de6e7e-ff17-4d4f-a08b-c0a567f17a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:42:04.875819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de6e7e-ff17-4d4f-a08b-c0a567f17a4d None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:04.876098 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de6e7e-ff17-4d4f-a08b-c0a567f17a4d None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:04.876368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de6e7e-ff17-4d4f-a08b-c0a567f17a4d None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:04.876957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de6e7e-ff17-4d4f-a08b-c0a567f17a4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:42:04.878600 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de6e7e-ff17-4d4f-a08b-c0a567f17a4d None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:42:04.885109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 651/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Aug 18 14:42:04 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:42:06.726780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc766016-2814-42b8-ad4c-7b716f9817f4 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:06.727341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc766016-2814-42b8-ad4c-7b716f9817f4 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:06.727341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc766016-2814-42b8-ad4c-7b716f9817f4 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:06.728674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 651/1302] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:42:06.734213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed2fe2-495b-46d4-99e9-7509758be316 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:06.734582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed2fe2-495b-46d4-99e9-7509758be316 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:06.734772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed2fe2-495b-46d4-99e9-7509758be316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:06.753480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f9ed2fe2-495b-46d4-99e9-7509758be316 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:42:06.754678 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f9ed2fe2-495b-46d4-99e9-7509758be316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:42:06.818191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 652/1303] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:42:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 14:42:07.083997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc766016-2814-42b8-ad4c-7b716f9817f4 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:42:07.176126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:42:07.182826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc766016-2814-42b8-ad4c-7b716f9817f4 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:42:07.215242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329009c4-db95-4777-951f-195c7a44b94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:42:07.215876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329009c4-db95-4777-951f-195c7a44b94c None admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:07.216155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329009c4-db95-4777-951f-195c7a44b94c None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:07.216424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329009c4-db95-4777-951f-195c7a44b94c None admin] PATH_INFO: `/v3/services/placement` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:07.216998 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329009c4-db95-4777-951f-195c7a44b94c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:42:07.218357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329009c4-db95-4777-951f-195c7a44b94c None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application [None req-329009c4-db95-4777-951f-195c7a44b94c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 14:42:07.226713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 14:42:07.227914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application with Lock( Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self._enter() Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application return self.creator() Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application created_value = creator( Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: ERROR keystone.server.flask.application Aug 18 14:42:07.228943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 652/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Aug 18 14:42:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 14:42:07.234349 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed2fe2-495b-46d4-99e9-7509758be316 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:42:07.306530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:42:07.328924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed2fe2-495b-46d4-99e9-7509758be316 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:42:07.361320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02a73d-29a5-4869-90bf-e4fc7f343114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:42:07.361873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02a73d-29a5-4869-90bf-e4fc7f343114 None admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:07.362176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02a73d-29a5-4869-90bf-e4fc7f343114 None admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:07.362618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02a73d-29a5-4869-90bf-e4fc7f343114 None admin] PATH_INFO: `/v3/services` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:07.363171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02a73d-29a5-4869-90bf-e4fc7f343114 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:42:07.363472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02a73d-29a5-4869-90bf-e4fc7f343114 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:42:07.365014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02a73d-29a5-4869-90bf-e4fc7f343114 None admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:42:07.370637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 653/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Thu Aug 18 14:42:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:42:07.377769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329009c4-db95-4777-951f-195c7a44b94c None admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:42:07.476112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329009c4-db95-4777-951f-195c7a44b94c None admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:42:07.509360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4374f7-e8ed-4912-94e3-fde1282ee9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:42:07.509948 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4374f7-e8ed-4912-94e3-fde1282ee9b7 None admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:42:07.510231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4374f7-e8ed-4912-94e3-fde1282ee9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:42:07.510499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4374f7-e8ed-4912-94e3-fde1282ee9b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:42:07.511140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4374f7-e8ed-4912-94e3-fde1282ee9b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:42:07.512562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4374f7-e8ed-4912-94e3-fde1282ee9b7 None admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:42:07.531171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 653/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:42:07 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:42:17.367711 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=30 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:42:17.527305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:10.373715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a4d3f-fc52-4db9-9da9-ee6216e2e5d2 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:10.374329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a4d3f-fc52-4db9-9da9-ee6216e2e5d2 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:10.374329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a4d3f-fc52-4db9-9da9-ee6216e2e5d2 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:10.381854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22d74e-cd6f-49f7-91bb-7de3c75dd350 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:10.382408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22d74e-cd6f-49f7-91bb-7de3c75dd350 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:10.382408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22d74e-cd6f-49f7-91bb-7de3c75dd350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:10.388502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 654/1307] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:10 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:10.465637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-6a22d74e-cd6f-49f7-91bb-7de3c75dd350 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:10.466858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6a22d74e-cd6f-49f7-91bb-7de3c75dd350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:10.587731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 654/1308] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:43:10.919547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ef971-2174-446b-8f1f-81cea207bc1e None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:10.920111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ef971-2174-446b-8f1f-81cea207bc1e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:10.920609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ef971-2174-446b-8f1f-81cea207bc1e None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:10.922514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 655/1309] 10.222.0.36 () {54 vars in 965 bytes} [Thu Aug 18 14:43:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:10.930813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e84b9-1c2d-470a-9768-cfb2ea67e507 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:10.931102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e84b9-1c2d-470a-9768-cfb2ea67e507 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:10.931375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e84b9-1c2d-470a-9768-cfb2ea67e507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:11.054505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-459e84b9-1c2d-470a-9768-cfb2ea67e507 None None] MFA Rules not processed for user `00f209d8426a49acaca7e9d00a1b9ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:11.056088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-459e84b9-1c2d-470a-9768-cfb2ea67e507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:11.083594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:11.185465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 655/1310] 10.222.0.36 () {58 vars in 1062 bytes} [Thu Aug 18 14:43:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:43:11.186644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ef971-2174-446b-8f1f-81cea207bc1e None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:11.188659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-731ef971-2174-446b-8f1f-81cea207bc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:11.333041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ef971-2174-446b-8f1f-81cea207bc1e None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:11.429759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca361cb-7235-44bf-971e-a5a97b8517ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:11.431132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca361cb-7235-44bf-971e-a5a97b8517ca service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:11.431558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca361cb-7235-44bf-971e-a5a97b8517ca service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:11.432002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca361cb-7235-44bf-971e-a5a97b8517ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:11.434482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bca361cb-7235-44bf-971e-a5a97b8517ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:11.492093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:11.557096 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca361cb-7235-44bf-971e-a5a97b8517ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:11.559079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca361cb-7235-44bf-971e-a5a97b8517ca service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:11.580968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 656/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:43:13.794870 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805abd8-c537-45dc-a924-1a7396183a44 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:13.795444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805abd8-c537-45dc-a924-1a7396183a44 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:13.795444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805abd8-c537-45dc-a924-1a7396183a44 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:13.796951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 656/1312] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:13.801812 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7c6c3-51e2-4f53-a73c-4292e691dd5b None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:13.802202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7c6c3-51e2-4f53-a73c-4292e691dd5b None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:13.802352 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7c6c3-51e2-4f53-a73c-4292e691dd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:13.883277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-afd7c6c3-51e2-4f53-a73c-4292e691dd5b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:13.884497 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-afd7c6c3-51e2-4f53-a73c-4292e691dd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:13.981171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 657/1313] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:43:14.260409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805abd8-c537-45dc-a924-1a7396183a44 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:14.262039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5805abd8-c537-45dc-a924-1a7396183a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:14.364871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805abd8-c537-45dc-a924-1a7396183a44 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:14.441974 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:14.445404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9496e-3f66-4ed6-8295-bcddc7f6535a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:14.446063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9496e-3f66-4ed6-8295-bcddc7f6535a service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:14.446575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9496e-3f66-4ed6-8295-bcddc7f6535a service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:14.447053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9496e-3f66-4ed6-8295-bcddc7f6535a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:14.449379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-49c9496e-3f66-4ed6-8295-bcddc7f6535a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:14.558228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9496e-3f66-4ed6-8295-bcddc7f6535a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:14.559673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9496e-3f66-4ed6-8295-bcddc7f6535a service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:14.577800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 657/1314] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:43:16.759016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76cc17-d88f-4e57-b224-f1c7d55ddcde None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:16.759525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76cc17-d88f-4e57-b224-f1c7d55ddcde None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:16.759525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76cc17-d88f-4e57-b224-f1c7d55ddcde None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:16.760889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 658/1315] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:16.766073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceed601-b410-4bbf-84fb-5f6e00d8bfae None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:16.766540 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceed601-b410-4bbf-84fb-5f6e00d8bfae None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:16.766633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceed601-b410-4bbf-84fb-5f6e00d8bfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:16.831223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-2ceed601-b410-4bbf-84fb-5f6e00d8bfae None None] MFA Rules not processed for user `a485edf4c68a4d0eaed5f79e91691cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:16.832458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2ceed601-b410-4bbf-84fb-5f6e00d8bfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:16.883563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:16.928568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 658/1316] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:43:17.215162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76cc17-d88f-4e57-b224-f1c7d55ddcde None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:17.216743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9c76cc17-d88f-4e57-b224-f1c7d55ddcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:17.226378 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:17.318854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76cc17-d88f-4e57-b224-f1c7d55ddcde None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:17.392187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a16ce3-ca1d-4ca6-8cc0-d209aa982b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:17.392793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a16ce3-ca1d-4ca6-8cc0-d209aa982b98 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:17.393072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a16ce3-ca1d-4ca6-8cc0-d209aa982b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:17.393344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a16ce3-ca1d-4ca6-8cc0-d209aa982b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:17.394943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-24a16ce3-ca1d-4ca6-8cc0-d209aa982b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:17.455449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:17.501883 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a16ce3-ca1d-4ca6-8cc0-d209aa982b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:17.503238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a16ce3-ca1d-4ca6-8cc0-d209aa982b98 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:17.521439 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 659/1317] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:17 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:43:19.975993 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d732194-a4db-4794-8e41-cdb10720ebb2 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:19.976503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d732194-a4db-4794-8e41-cdb10720ebb2 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:19.976624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d732194-a4db-4794-8e41-cdb10720ebb2 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:19.977891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 659/1318] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:19.982993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45af450-7cbf-436a-9e62-034e8c684954 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:19.983438 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45af450-7cbf-436a-9e62-034e8c684954 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:19.983534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45af450-7cbf-436a-9e62-034e8c684954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:20.047049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-b45af450-7cbf-436a-9e62-034e8c684954 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:20.048271 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b45af450-7cbf-436a-9e62-034e8c684954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:20.140067 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 660/1319] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:43:20.425203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d732194-a4db-4794-8e41-cdb10720ebb2 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:20.426751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5d732194-a4db-4794-8e41-cdb10720ebb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:20.530581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d732194-a4db-4794-8e41-cdb10720ebb2 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:20.604388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc27d6-1e0c-42eb-b8f1-6ad352e61ee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:20.605546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc27d6-1e0c-42eb-b8f1-6ad352e61ee9 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:20.605818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc27d6-1e0c-42eb-b8f1-6ad352e61ee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:20.606084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc27d6-1e0c-42eb-b8f1-6ad352e61ee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:20.607666 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-72dc27d6-1e0c-42eb-b8f1-6ad352e61ee9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:20.614891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:20.712851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc27d6-1e0c-42eb-b8f1-6ad352e61ee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:20.714307 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc27d6-1e0c-42eb-b8f1-6ad352e61ee9 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:20.732097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 660/1320] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:43:23.274585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c22b7-5d1a-4492-b56e-8827332a17a0 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:23.275162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c22b7-5d1a-4492-b56e-8827332a17a0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:23.275162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c22b7-5d1a-4492-b56e-8827332a17a0 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:23.276674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 661/1321] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:23.281684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20123aaa-bc56-4c02-9f4c-27e3ecb7f145 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:23.282110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20123aaa-bc56-4c02-9f4c-27e3ecb7f145 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:23.282229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20123aaa-bc56-4c02-9f4c-27e3ecb7f145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:23.347665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-20123aaa-bc56-4c02-9f4c-27e3ecb7f145 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:23.348881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-20123aaa-bc56-4c02-9f4c-27e3ecb7f145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:23.406224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:23.445263 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 661/1322] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:43:23.747969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb42f0-1085-41f0-a583-8606d777618a None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:23.748255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb42f0-1085-41f0-a583-8606d777618a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:23.748526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb42f0-1085-41f0-a583-8606d777618a None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:23.749858 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 662/1323] 10.222.0.36 () {54 vars in 965 bytes} [Thu Aug 18 14:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:23.756264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d66e3-702e-4a6b-af10-090a4c074e54 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:23.756545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d66e3-702e-4a6b-af10-090a4c074e54 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:23.756815 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d66e3-702e-4a6b-af10-090a4c074e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:23.830402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-5b0d66e3-702e-4a6b-af10-090a4c074e54 None None] MFA Rules not processed for user `00f209d8426a49acaca7e9d00a1b9ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:23.831598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5b0d66e3-702e-4a6b-af10-090a4c074e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:23.907043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 662/1324] 10.222.0.36 () {58 vars in 1062 bytes} [Thu Aug 18 14:43:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:43:23.915970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb42f0-1085-41f0-a583-8606d777618a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:23.917999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-64cb42f0-1085-41f0-a583-8606d777618a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:23.938125 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:24.020354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb42f0-1085-41f0-a583-8606d777618a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:24.092383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8acb5-69be-46ff-847c-6cdeb50ea1f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:24.092987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8acb5-69be-46ff-847c-6cdeb50ea1f4 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:24.093265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8acb5-69be-46ff-847c-6cdeb50ea1f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:24.093534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8acb5-69be-46ff-847c-6cdeb50ea1f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:24.095146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-fff8acb5-69be-46ff-847c-6cdeb50ea1f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:24.165314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:24.203392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8acb5-69be-46ff-847c-6cdeb50ea1f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:24.204874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8acb5-69be-46ff-847c-6cdeb50ea1f4 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:24.223395 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 663/1325] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:43:26.379593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ff1e2-682e-4a39-988a-0f1561e8b6cf None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:26.380175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ff1e2-682e-4a39-988a-0f1561e8b6cf None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:26.380175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ff1e2-682e-4a39-988a-0f1561e8b6cf None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:26.381718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 663/1326] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:26.387325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7dee4-431c-4b76-a85f-700b0fcea477 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:26.387841 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7dee4-431c-4b76-a85f-700b0fcea477 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:26.387955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7dee4-431c-4b76-a85f-700b0fcea477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:26.453542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-05b7dee4-431c-4b76-a85f-700b0fcea477 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:26.454776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-05b7dee4-431c-4b76-a85f-700b0fcea477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:26.550238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 664/1327] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:43:26.832056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ff1e2-682e-4a39-988a-0f1561e8b6cf None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:26.833627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-226ff1e2-682e-4a39-988a-0f1561e8b6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:26.895106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:26.939174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ff1e2-682e-4a39-988a-0f1561e8b6cf None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:27.012630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c89c64-267d-45c2-b7fc-565096201115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:27.013227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c89c64-267d-45c2-b7fc-565096201115 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:27.013500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c89c64-267d-45c2-b7fc-565096201115 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:27.013763 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c89c64-267d-45c2-b7fc-565096201115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:27.015339 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d4c89c64-267d-45c2-b7fc-565096201115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:27.121605 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:27.123914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c89c64-267d-45c2-b7fc-565096201115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:27.126020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c89c64-267d-45c2-b7fc-565096201115 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:27.144191 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 664/1328] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:43:29.079500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5087dc3-91e6-4e2a-a1ac-015b065ca9b8 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:29.080024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5087dc3-91e6-4e2a-a1ac-015b065ca9b8 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:29.080119 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5087dc3-91e6-4e2a-a1ac-015b065ca9b8 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:29.081401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 665/1329] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:29.086457 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f42951-de2a-4bdf-816e-ff736e51ddfb None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:29.086866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f42951-de2a-4bdf-816e-ff736e51ddfb None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:29.087156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f42951-de2a-4bdf-816e-ff736e51ddfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:29.167825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-32f42951-de2a-4bdf-816e-ff736e51ddfb None None] MFA Rules not processed for user `a485edf4c68a4d0eaed5f79e91691cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:29.169078 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-32f42951-de2a-4bdf-816e-ff736e51ddfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:29.272355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 665/1330] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:29 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:43:29.555369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5087dc3-91e6-4e2a-a1ac-015b065ca9b8 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:29.556978 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f5087dc3-91e6-4e2a-a1ac-015b065ca9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:29.588550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:29.661956 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5087dc3-91e6-4e2a-a1ac-015b065ca9b8 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:29.733941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43df7e-e537-489b-8b07-83cab6877222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:29.734548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43df7e-e537-489b-8b07-83cab6877222 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:29.734825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43df7e-e537-489b-8b07-83cab6877222 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:29.735096 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43df7e-e537-489b-8b07-83cab6877222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:29.736743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bb43df7e-e537-489b-8b07-83cab6877222 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:29.818916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:29.852089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43df7e-e537-489b-8b07-83cab6877222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:29.853529 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43df7e-e537-489b-8b07-83cab6877222 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:29.873447 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 666/1331] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:43:32.156462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85d041-d2a7-4611-8c03-829bd94b0b2a None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:32.156979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85d041-d2a7-4611-8c03-829bd94b0b2a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:32.156979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85d041-d2a7-4611-8c03-829bd94b0b2a None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:32.158258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 666/1332] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:32.163284 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678adc1a-565a-4a4e-af70-8760f70bba0c None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:32.163665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678adc1a-565a-4a4e-af70-8760f70bba0c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:32.163857 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678adc1a-565a-4a4e-af70-8760f70bba0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:32.227559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-678adc1a-565a-4a4e-af70-8760f70bba0c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:32.228801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-678adc1a-565a-4a4e-af70-8760f70bba0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:32.322161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 667/1333] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:43:32.600846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85d041-d2a7-4611-8c03-829bd94b0b2a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:32.602416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5e85d041-d2a7-4611-8c03-829bd94b0b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:32.673052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:32.706818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85d041-d2a7-4611-8c03-829bd94b0b2a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:32.778530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4134d6-fe19-42cf-91d8-63f670b3c125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:32.779120 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4134d6-fe19-42cf-91d8-63f670b3c125 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:32.779392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4134d6-fe19-42cf-91d8-63f670b3c125 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:32.779657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4134d6-fe19-42cf-91d8-63f670b3c125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:32.781254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ea4134d6-fe19-42cf-91d8-63f670b3c125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:32.885479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4134d6-fe19-42cf-91d8-63f670b3c125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:32.887406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4134d6-fe19-42cf-91d8-63f670b3c125 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:32.897393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:32.906140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 667/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:43:34.846080 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329d4ad-34e0-4039-bd49-14ec8ac4a97e None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:34.846592 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329d4ad-34e0-4039-bd49-14ec8ac4a97e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:34.846592 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329d4ad-34e0-4039-bd49-14ec8ac4a97e None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:34.847914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 668/1335] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:34.853072 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe0ab3-7543-42e9-b381-1024b3b0243b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:34.853468 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe0ab3-7543-42e9-b381-1024b3b0243b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:34.853721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe0ab3-7543-42e9-b381-1024b3b0243b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:34.919634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-dbfe0ab3-7543-42e9-b381-1024b3b0243b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:34.920890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-dbfe0ab3-7543-42e9-b381-1024b3b0243b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:35.015543 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 668/1336] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:43:35.313830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329d4ad-34e0-4039-bd49-14ec8ac4a97e None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:35.315383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6329d4ad-34e0-4039-bd49-14ec8ac4a97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:35.355340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:35.416561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329d4ad-34e0-4039-bd49-14ec8ac4a97e None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:35.487808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bdd5b-9bd8-40fa-8844-b820f0d063a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:35.488402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bdd5b-9bd8-40fa-8844-b820f0d063a6 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:35.488677 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bdd5b-9bd8-40fa-8844-b820f0d063a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:35.488946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bdd5b-9bd8-40fa-8844-b820f0d063a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:35.490521 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-685bdd5b-9bd8-40fa-8844-b820f0d063a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:35.572300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:35.597863 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bdd5b-9bd8-40fa-8844-b820f0d063a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:35.599303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bdd5b-9bd8-40fa-8844-b820f0d063a6 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:35.618656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 669/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:43:37.882274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc8440-c918-42f3-8f96-b25a0e1fa617 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:37.882873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc8440-c918-42f3-8f96-b25a0e1fa617 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:37.882873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc8440-c918-42f3-8f96-b25a0e1fa617 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:37.884196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 669/1338] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:37.889225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b899947-9b8b-457d-9d7a-56a325a677d4 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:37.889674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b899947-9b8b-457d-9d7a-56a325a677d4 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:37.889778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b899947-9b8b-457d-9d7a-56a325a677d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:37.955009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-0b899947-9b8b-457d-9d7a-56a325a677d4 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:37.956246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-0b899947-9b8b-457d-9d7a-56a325a677d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:38.050010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 670/1339] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:43:38.328672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc8440-c918-42f3-8f96-b25a0e1fa617 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:38.330297 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-52cc8440-c918-42f3-8f96-b25a0e1fa617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:38.410165 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:38.438145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc8440-c918-42f3-8f96-b25a0e1fa617 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:38.509750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20641896-3200-4e03-9172-d6c0bd9af205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:38.510352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20641896-3200-4e03-9172-d6c0bd9af205 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:38.511182 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20641896-3200-4e03-9172-d6c0bd9af205 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:38.511452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20641896-3200-4e03-9172-d6c0bd9af205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:38.513068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-20641896-3200-4e03-9172-d6c0bd9af205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:38.619222 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20641896-3200-4e03-9172-d6c0bd9af205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:38.621128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20641896-3200-4e03-9172-d6c0bd9af205 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:38.639276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 670/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:43:42.497948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d26176-101c-4dad-b85a-458925b73c07 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:42.498495 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d26176-101c-4dad-b85a-458925b73c07 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:42.498495 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d26176-101c-4dad-b85a-458925b73c07 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:42.499983 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 671/1341] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:42.505774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee1a2f-2723-40f3-a37e-0bc57efff57c None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:42.506253 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee1a2f-2723-40f3-a37e-0bc57efff57c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:42.506349 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee1a2f-2723-40f3-a37e-0bc57efff57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:42.512247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:42.573792 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-30ee1a2f-2723-40f3-a37e-0bc57efff57c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:42.574905 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-30ee1a2f-2723-40f3-a37e-0bc57efff57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:42.669512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 671/1342] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:43:42.951330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d26176-101c-4dad-b85a-458925b73c07 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:42.952938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-07d26176-101c-4dad-b85a-458925b73c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:43.002078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:43.063253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d26176-101c-4dad-b85a-458925b73c07 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:43.136020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406bf8b-fe82-4766-b306-a74b972a9774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:43.136542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406bf8b-fe82-4766-b306-a74b972a9774 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:43.136816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406bf8b-fe82-4766-b306-a74b972a9774 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:43.137082 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406bf8b-fe82-4766-b306-a74b972a9774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:43.138655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5406bf8b-fe82-4766-b306-a74b972a9774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:43.227999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:43.244323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406bf8b-fe82-4766-b306-a74b972a9774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:43.245758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406bf8b-fe82-4766-b306-a74b972a9774 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:43.264827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 672/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:43:45.998451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467ea31-8700-4b68-8196-cc4624601873 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:45.998983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467ea31-8700-4b68-8196-cc4624601873 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:45.998983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467ea31-8700-4b68-8196-cc4624601873 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:46.000375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 672/1344] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:46.005756 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553cedd-564b-4197-9b89-52bcf2072139 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:46.006169 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553cedd-564b-4197-9b89-52bcf2072139 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:46.006303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553cedd-564b-4197-9b89-52bcf2072139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:46.072633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f553cedd-564b-4197-9b89-52bcf2072139 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:46.073788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f553cedd-564b-4197-9b89-52bcf2072139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:46.169973 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 673/1345] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:43:46.449835 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467ea31-8700-4b68-8196-cc4624601873 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:46.451381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c467ea31-8700-4b68-8196-cc4624601873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:46.544142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:46.561504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467ea31-8700-4b68-8196-cc4624601873 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:46.634553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32374aa0-0f17-43ef-a629-496ba126a46d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:46.635164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32374aa0-0f17-43ef-a629-496ba126a46d service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:46.636036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32374aa0-0f17-43ef-a629-496ba126a46d service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:46.636285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32374aa0-0f17-43ef-a629-496ba126a46d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:46.637905 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-32374aa0-0f17-43ef-a629-496ba126a46d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:46.750139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32374aa0-0f17-43ef-a629-496ba126a46d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:46.751592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32374aa0-0f17-43ef-a629-496ba126a46d service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:46.771109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 673/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:43:50.830498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d289f-560a-4a6c-8f75-cf33c5bb7bd4 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:50.831061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d289f-560a-4a6c-8f75-cf33c5bb7bd4 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:50.831061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d289f-560a-4a6c-8f75-cf33c5bb7bd4 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:50.836292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 674/1347] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:43:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:50.838023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aada32-1022-41b7-a7be-622e2e6bb580 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:50.839035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aada32-1022-41b7-a7be-622e2e6bb580 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:50.839230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aada32-1022-41b7-a7be-622e2e6bb580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:50.859331 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:50.926506 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-27aada32-1022-41b7-a7be-622e2e6bb580 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:50.927813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-27aada32-1022-41b7-a7be-622e2e6bb580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:51.029925 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 674/1348] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:43:51.310894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d289f-560a-4a6c-8f75-cf33c5bb7bd4 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:43:51.312488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c63d289f-560a-4a6c-8f75-cf33c5bb7bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:51.368950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:51.414611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d289f-560a-4a6c-8f75-cf33c5bb7bd4 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:43:51.485793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361a1a0-cbc9-448d-9a73-03e12c53b401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:43:51.486397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a1a0-cbc9-448d-9a73-03e12c53b401 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:51.486675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a1a0-cbc9-448d-9a73-03e12c53b401 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:51.486947 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a1a0-cbc9-448d-9a73-03e12c53b401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:51.488574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e361a1a0-cbc9-448d-9a73-03e12c53b401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:51.591750 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:43:51.595860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361a1a0-cbc9-448d-9a73-03e12c53b401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:43:51.597300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361a1a0-cbc9-448d-9a73-03e12c53b401 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:43:51.615247 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 675/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:43:56.480374 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594878a7-0320-4aec-9610-839e9cda8f89 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:56.480766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594878a7-0320-4aec-9610-839e9cda8f89 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:56.480843 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594878a7-0320-4aec-9610-839e9cda8f89 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:56.482157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 675/1350] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 14:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:43:56.487731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a3813-b1e6-4228-b6a0-09d193b39264 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:43:56.488159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a3813-b1e6-4228-b6a0-09d193b39264 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:43:56.488333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a3813-b1e6-4228-b6a0-09d193b39264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:43:56.630109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-429a3813-b1e6-4228-b6a0-09d193b39264 None None] MFA Rules not processed for user `db581b10d78a4f9eae33fadce0d92b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:43:56.631622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-429a3813-b1e6-4228-b6a0-09d193b39264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:43:56.750360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 676/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:00.803880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd382af-eeca-4a0a-9048-8a02ec7099e5 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:00.804728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd382af-eeca-4a0a-9048-8a02ec7099e5 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:00.805140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd382af-eeca-4a0a-9048-8a02ec7099e5 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:00.807447 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 676/1352] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 14:44:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:00.812790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13313118-71fa-4c67-8f80-d2818b7c86e4 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:00.813530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13313118-71fa-4c67-8f80-d2818b7c86e4 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:00.813954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13313118-71fa-4c67-8f80-d2818b7c86e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:00.871289 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:00.899196 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-13313118-71fa-4c67-8f80-d2818b7c86e4 None None] MFA Rules not processed for user `db581b10d78a4f9eae33fadce0d92b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:00.901108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-13313118-71fa-4c67-8f80-d2818b7c86e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:01.018579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 677/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:01.028008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=59 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:02.456610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0a30d-ad64-47ea-95b7-8bb72739c223 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:02.457271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0a30d-ad64-47ea-95b7-8bb72739c223 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:02.457271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0a30d-ad64-47ea-95b7-8bb72739c223 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:02.458549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 677/1354] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 14:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:02.464633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2c198-7a63-4fb9-958e-ca826f60d125 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:02.464633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2c198-7a63-4fb9-958e-ca826f60d125 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:02.465095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2c198-7a63-4fb9-958e-ca826f60d125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:02.539617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-61d2c198-7a63-4fb9-958e-ca826f60d125 None None] MFA Rules not processed for user `db581b10d78a4f9eae33fadce0d92b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:02.541855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-61d2c198-7a63-4fb9-958e-ca826f60d125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:02.584165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:02.649115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 678/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 14:44:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:44:05.898068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eb39a-42e9-478e-aaee-6c6ada0bd22f None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:05.898531 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eb39a-42e9-478e-aaee-6c6ada0bd22f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:05.898635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eb39a-42e9-478e-aaee-6c6ada0bd22f None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:05.903745 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 678/1356] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:05.905832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66329359-e9f0-46f7-b211-0482890f5c35 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:05.906252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66329359-e9f0-46f7-b211-0482890f5c35 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:05.906397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66329359-e9f0-46f7-b211-0482890f5c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:06.003834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-66329359-e9f0-46f7-b211-0482890f5c35 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:06.006243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-66329359-e9f0-46f7-b211-0482890f5c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:06.120245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 679/1357] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:06.424999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf26263-ef03-46c7-adb6-8a8eb6eb2111 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:06.425289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf26263-ef03-46c7-adb6-8a8eb6eb2111 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:06.425581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf26263-ef03-46c7-adb6-8a8eb6eb2111 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:06.427260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 679/1358] 10.222.0.36 () {54 vars in 962 bytes} [Thu Aug 18 14:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:06.436257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb17b9-19be-452d-baa3-e5e7dbcafffb None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:06.436791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb17b9-19be-452d-baa3-e5e7dbcafffb None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:06.437508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb17b9-19be-452d-baa3-e5e7dbcafffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:06.457575 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:06.524234 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-8bbb17b9-19be-452d-baa3-e5e7dbcafffb None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:06.526163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8bbb17b9-19be-452d-baa3-e5e7dbcafffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:06.658969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 680/1359] 10.222.0.36 () {58 vars in 1059 bytes} [Thu Aug 18 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:06.666875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf26263-ef03-46c7-adb6-8a8eb6eb2111 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:06.668518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ebf26263-ef03-46c7-adb6-8a8eb6eb2111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:06.819644 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf26263-ef03-46c7-adb6-8a8eb6eb2111 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:06.899465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:06.948110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef596e-a0f2-4c18-8434-3b7e6ba4c79b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:06.949018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef596e-a0f2-4c18-8434-3b7e6ba4c79b service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:06.949289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef596e-a0f2-4c18-8434-3b7e6ba4c79b service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:06.949554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef596e-a0f2-4c18-8434-3b7e6ba4c79b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:06.951136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a1ef596e-a0f2-4c18-8434-3b7e6ba4c79b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:07.044766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef596e-a0f2-4c18-8434-3b7e6ba4c79b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:07.046541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef596e-a0f2-4c18-8434-3b7e6ba4c79b service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:07.065417 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 680/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:09.171714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4ca45-d84f-49f7-877c-1e7e19508fd2 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:09.172251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4ca45-d84f-49f7-877c-1e7e19508fd2 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:09.173107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4ca45-d84f-49f7-877c-1e7e19508fd2 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:09.174587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 681/1361] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:09.180092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b746ab6-8198-4699-91d2-33c728efe2d6 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:09.180911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b746ab6-8198-4699-91d2-33c728efe2d6 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:09.181729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b746ab6-8198-4699-91d2-33c728efe2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:09.250583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-5b746ab6-8198-4699-91d2-33c728efe2d6 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:09.251831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5b746ab6-8198-4699-91d2-33c728efe2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:09.291517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:09.367004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 681/1362] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:09.649016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4ca45-d84f-49f7-877c-1e7e19508fd2 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:09.651107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c2e4ca45-d84f-49f7-877c-1e7e19508fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:09.728608 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:09.818428 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4ca45-d84f-49f7-877c-1e7e19508fd2 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:09.953603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96fe1c-4d0e-4213-9de8-b1c1e8ab5627 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:09.954217 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96fe1c-4d0e-4213-9de8-b1c1e8ab5627 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:09.954499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96fe1c-4d0e-4213-9de8-b1c1e8ab5627 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:09.954781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96fe1c-4d0e-4213-9de8-b1c1e8ab5627 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:09.956436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2d96fe1c-4d0e-4213-9de8-b1c1e8ab5627 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:09.980566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:10.062585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96fe1c-4d0e-4213-9de8-b1c1e8ab5627 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:10.064520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96fe1c-4d0e-4213-9de8-b1c1e8ab5627 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:10.085891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 682/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:12.300990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa74033-e339-4305-9116-1c5eb48ec254 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:12.301553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa74033-e339-4305-9116-1c5eb48ec254 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:12.301553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa74033-e339-4305-9116-1c5eb48ec254 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:12.302851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 682/1364] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:12.310201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807a4df-db6a-4ca1-a79e-4d12046cf66e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:12.310703 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807a4df-db6a-4ca1-a79e-4d12046cf66e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:12.310811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807a4df-db6a-4ca1-a79e-4d12046cf66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:12.406122 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-d807a4df-db6a-4ca1-a79e-4d12046cf66e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:12.407536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d807a4df-db6a-4ca1-a79e-4d12046cf66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:12.508702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:12.520321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 683/1365] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:12.809084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6382f1-3675-427f-b907-ef342e879f4f None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:12.809498 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6382f1-3675-427f-b907-ef342e879f4f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:12.809897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6382f1-3675-427f-b907-ef342e879f4f None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:12.811575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 683/1366] 10.222.0.36 () {54 vars in 962 bytes} [Thu Aug 18 14:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:12.816982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829500fc-4eac-4c5b-b07f-1a7fb1748b30 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:12.817360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829500fc-4eac-4c5b-b07f-1a7fb1748b30 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:12.817705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829500fc-4eac-4c5b-b07f-1a7fb1748b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:12.919753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-829500fc-4eac-4c5b-b07f-1a7fb1748b30 None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:12.919959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-829500fc-4eac-4c5b-b07f-1a7fb1748b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:13.055704 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 684/1367] 10.222.0.36 () {58 vars in 1059 bytes} [Thu Aug 18 14:44:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:13.064139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6382f1-3675-427f-b907-ef342e879f4f None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:13.065803 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2d6382f1-3675-427f-b907-ef342e879f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:13.219863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6382f1-3675-427f-b907-ef342e879f4f None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:13.242639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:13.342111 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197f138-cfe8-47ef-a94b-3fff17da7f30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:13.343929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197f138-cfe8-47ef-a94b-3fff17da7f30 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:13.344425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197f138-cfe8-47ef-a94b-3fff17da7f30 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:13.344920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197f138-cfe8-47ef-a94b-3fff17da7f30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:13.346723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6197f138-cfe8-47ef-a94b-3fff17da7f30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:13.443852 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197f138-cfe8-47ef-a94b-3fff17da7f30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:13.446057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197f138-cfe8-47ef-a94b-3fff17da7f30 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:13.470332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 684/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:14.062314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a77f8-301a-44d5-8aed-3405c395354d None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:14.062841 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a77f8-301a-44d5-8aed-3405c395354d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:14.062841 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a77f8-301a-44d5-8aed-3405c395354d None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:14.064352 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 685/1369] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:14.069482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b248c-d9f2-45e0-86a4-6c3f938edfe8 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:14.069773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b248c-d9f2-45e0-86a4-6c3f938edfe8 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:14.070047 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b248c-d9f2-45e0-86a4-6c3f938edfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:14.104978 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:14.143173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-371b248c-d9f2-45e0-86a4-6c3f938edfe8 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:14.144426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-371b248c-d9f2-45e0-86a4-6c3f938edfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:14.244401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 685/1370] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:14.831917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c531-b2f6-4cac-831c-9a1068e732cb None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:14.831917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c531-b2f6-4cac-831c-9a1068e732cb None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:14.831917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c531-b2f6-4cac-831c-9a1068e732cb None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:14.831917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 686/1371] 10.222.0.36 () {54 vars in 964 bytes} [Thu Aug 18 14:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:14.835401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a16786-6b76-4e18-87d5-d5a5e9350be4 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:14.835898 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a16786-6b76-4e18-87d5-d5a5e9350be4 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:14.836060 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a16786-6b76-4e18-87d5-d5a5e9350be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:14.922939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-34a16786-6b76-4e18-87d5-d5a5e9350be4 None None] MFA Rules not processed for user `1a9b29546bc848ef9e92b3d267da18ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:14.924972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-34a16786-6b76-4e18-87d5-d5a5e9350be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:14.973977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:15.005749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 686/1372] 10.222.0.36 () {58 vars in 1061 bytes} [Thu Aug 18 14:44:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:15.013994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c531-b2f6-4cac-831c-9a1068e732cb None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:15.015632 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d159c531-b2f6-4cac-831c-9a1068e732cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:15.068056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:15.124300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c531-b2f6-4cac-831c-9a1068e732cb None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:15.207163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93116669-c7e5-4f48-9578-faaff630a808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:15.207960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93116669-c7e5-4f48-9578-faaff630a808 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:15.208354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93116669-c7e5-4f48-9578-faaff630a808 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:15.208731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93116669-c7e5-4f48-9578-faaff630a808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:15.210675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-93116669-c7e5-4f48-9578-faaff630a808 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:15.337692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:15.356603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93116669-c7e5-4f48-9578-faaff630a808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:15.358542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93116669-c7e5-4f48-9578-faaff630a808 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:15.385992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 687/1373] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:15.457847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08a84a-4aaa-428e-87f6-6b7f18b54f50 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:15.458138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08a84a-4aaa-428e-87f6-6b7f18b54f50 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:15.458415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08a84a-4aaa-428e-87f6-6b7f18b54f50 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:15.460816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 687/1374] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:15.466906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec25411-4c7a-4e34-b151-976842613a39 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:15.467460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec25411-4c7a-4e34-b151-976842613a39 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:15.468092 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec25411-4c7a-4e34-b151-976842613a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:15.552214 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-6ec25411-4c7a-4e34-b151-976842613a39 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:15.553452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6ec25411-4c7a-4e34-b151-976842613a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:15.656340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 688/1375] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:15.941142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08a84a-4aaa-428e-87f6-6b7f18b54f50 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:15.942773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6f08a84a-4aaa-428e-87f6-6b7f18b54f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:16.092591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08a84a-4aaa-428e-87f6-6b7f18b54f50 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:16.152775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:16.216970 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13189599-14f4-42fd-a972-3169b0d6f186 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:16.217522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13189599-14f4-42fd-a972-3169b0d6f186 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:16.217791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13189599-14f4-42fd-a972-3169b0d6f186 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:16.218048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13189599-14f4-42fd-a972-3169b0d6f186 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:16.219644 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-13189599-14f4-42fd-a972-3169b0d6f186 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:16.309287 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13189599-14f4-42fd-a972-3169b0d6f186 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:16.310901 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13189599-14f4-42fd-a972-3169b0d6f186 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:16.328789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 688/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:18.265799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c739010-0b8a-47d0-b4b7-a7a78a389711 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:18.266321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c739010-0b8a-47d0-b4b7-a7a78a389711 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:18.266416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c739010-0b8a-47d0-b4b7-a7a78a389711 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:18.267717 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 689/1377] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:18.273023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e0d0f-17bc-44f4-9b91-b0de8ee4ffed None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:18.273492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e0d0f-17bc-44f4-9b91-b0de8ee4ffed None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:18.273704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e0d0f-17bc-44f4-9b91-b0de8ee4ffed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:18.341269 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-f24e0d0f-17bc-44f4-9b91-b0de8ee4ffed None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:18.343858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f24e0d0f-17bc-44f4-9b91-b0de8ee4ffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:18.349503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:18.453925 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 689/1378] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:18.746906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c739010-0b8a-47d0-b4b7-a7a78a389711 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:18.748571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1c739010-0b8a-47d0-b4b7-a7a78a389711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:18.827882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:18.956326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c739010-0b8a-47d0-b4b7-a7a78a389711 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:19.089951 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f506b77-9db2-4e9c-bc0d-8e61adacd1e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:19.090492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f506b77-9db2-4e9c-bc0d-8e61adacd1e4 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:19.090772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f506b77-9db2-4e9c-bc0d-8e61adacd1e4 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:19.091046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f506b77-9db2-4e9c-bc0d-8e61adacd1e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:19.092675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4f506b77-9db2-4e9c-bc0d-8e61adacd1e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:19.111974 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:19.188462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f506b77-9db2-4e9c-bc0d-8e61adacd1e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:19.190068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f506b77-9db2-4e9c-bc0d-8e61adacd1e4 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:19.209628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 690/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:21.157484 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5263ef-6d3e-427b-ba40-603b9f9e962b None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:21.157940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5263ef-6d3e-427b-ba40-603b9f9e962b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:21.158024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5263ef-6d3e-427b-ba40-603b9f9e962b None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:21.159518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 690/1380] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:21.164670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d9df3-c258-40f3-bde7-aca6dec4423e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:21.165082 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d9df3-c258-40f3-bde7-aca6dec4423e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:21.165396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d9df3-c258-40f3-bde7-aca6dec4423e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:21.234972 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-426d9df3-c258-40f3-bde7-aca6dec4423e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:21.236039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-426d9df3-c258-40f3-bde7-aca6dec4423e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:21.343155 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:21.358599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 691/1381] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:21.640656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5263ef-6d3e-427b-ba40-603b9f9e962b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:21.642480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2f5263ef-6d3e-427b-ba40-603b9f9e962b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:21.802037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:21.814254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5263ef-6d3e-427b-ba40-603b9f9e962b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:21.947563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de251a-4a98-4f6a-9055-58f87325f83f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:21.948311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de251a-4a98-4f6a-9055-58f87325f83f service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:21.948586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de251a-4a98-4f6a-9055-58f87325f83f service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:21.948851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de251a-4a98-4f6a-9055-58f87325f83f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:21.950444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-86de251a-4a98-4f6a-9055-58f87325f83f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:22.054040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de251a-4a98-4f6a-9055-58f87325f83f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:22.056199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de251a-4a98-4f6a-9055-58f87325f83f service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:22.067456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:22.078185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 691/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:24.223126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f5a50-6378-413e-9969-7509e6246f5b None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:24.223728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f5a50-6378-413e-9969-7509e6246f5b None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:24.223728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f5a50-6378-413e-9969-7509e6246f5b None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:24.225115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 692/1383] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:24.230907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a8e77-ee5c-4d17-9a44-800ca0e0d645 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:24.231353 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a8e77-ee5c-4d17-9a44-800ca0e0d645 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:24.231590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a8e77-ee5c-4d17-9a44-800ca0e0d645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:24.310198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-b79a8e77-ee5c-4d17-9a44-800ca0e0d645 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:24.311471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b79a8e77-ee5c-4d17-9a44-800ca0e0d645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:24.412338 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 692/1384] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:24.710117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f5a50-6378-413e-9969-7509e6246f5b None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:24.712328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-660f5a50-6378-413e-9969-7509e6246f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:24.882993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f5a50-6378-413e-9969-7509e6246f5b None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:24.961335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:25.010265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af968bf2-79e8-4766-b8eb-4c56a15a2ef6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:25.011153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af968bf2-79e8-4766-b8eb-4c56a15a2ef6 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:25.011510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af968bf2-79e8-4766-b8eb-4c56a15a2ef6 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:25.011883 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af968bf2-79e8-4766-b8eb-4c56a15a2ef6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:25.014012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-af968bf2-79e8-4766-b8eb-4c56a15a2ef6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:25.112422 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af968bf2-79e8-4766-b8eb-4c56a15a2ef6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:25.114023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af968bf2-79e8-4766-b8eb-4c56a15a2ef6 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:25.132695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 693/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:27.162951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939933-5e81-446d-9a14-990a85baea6f None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:27.163484 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939933-5e81-446d-9a14-990a85baea6f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:27.163484 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939933-5e81-446d-9a14-990a85baea6f None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:27.165444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 693/1386] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:27.171230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffece6-027c-42d0-80fb-7621b1a44f0e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:27.171667 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffece6-027c-42d0-80fb-7621b1a44f0e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:27.171983 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffece6-027c-42d0-80fb-7621b1a44f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:27.241641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-23ffece6-027c-42d0-80fb-7621b1a44f0e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:27.242932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-23ffece6-027c-42d0-80fb-7621b1a44f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:27.277556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:27.344712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 694/1387] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:27.636758 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13939933-5e81-446d-9a14-990a85baea6f None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:27.638374 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-13939933-5e81-446d-9a14-990a85baea6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:27.726219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:27.797450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13939933-5e81-446d-9a14-990a85baea6f None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:27.913925 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7388bf6-0f1c-42cb-b2fb-27497988d008 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:27.914560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7388bf6-0f1c-42cb-b2fb-27497988d008 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:27.914887 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7388bf6-0f1c-42cb-b2fb-27497988d008 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:27.915111 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7388bf6-0f1c-42cb-b2fb-27497988d008 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:27.916766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e7388bf6-0f1c-42cb-b2fb-27497988d008 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:27.965034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:28.033079 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7388bf6-0f1c-42cb-b2fb-27497988d008 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:28.034743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7388bf6-0f1c-42cb-b2fb-27497988d008 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:28.062704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 694/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:44:30.048027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a5ff0-e5a5-4966-a13e-435a1b5efc11 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:30.048565 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a5ff0-e5a5-4966-a13e-435a1b5efc11 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:30.048565 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a5ff0-e5a5-4966-a13e-435a1b5efc11 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:30.049859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 695/1389] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:30.055377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadf597-bf3b-4ec3-bea8-516ee63d7c7b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:30.055858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadf597-bf3b-4ec3-bea8-516ee63d7c7b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:30.055983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadf597-bf3b-4ec3-bea8-516ee63d7c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:30.126729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8aadf597-bf3b-4ec3-bea8-516ee63d7c7b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:30.127967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8aadf597-bf3b-4ec3-bea8-516ee63d7c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:30.226037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 695/1390] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:30.518983 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a5ff0-e5a5-4966-a13e-435a1b5efc11 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:30.520584 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6e0a5ff0-e5a5-4966-a13e-435a1b5efc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:30.667451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a5ff0-e5a5-4966-a13e-435a1b5efc11 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:30.684138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:30.771879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76630b65-baa5-461b-9eca-d35bbf28c87e None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:30.772074 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76630b65-baa5-461b-9eca-d35bbf28c87e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:30.772336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76630b65-baa5-461b-9eca-d35bbf28c87e None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:30.773718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 696/1391] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:44:30.779425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca61c5-5f45-4c30-bfdb-174cbd34ae58 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:30.779723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca61c5-5f45-4c30-bfdb-174cbd34ae58 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:30.780004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca61c5-5f45-4c30-bfdb-174cbd34ae58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:30.782216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28b5de-17ba-4c1d-8f02-46bf50839e9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:30.782817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28b5de-17ba-4c1d-8f02-46bf50839e9e service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:30.783090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28b5de-17ba-4c1d-8f02-46bf50839e9e service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:30.783360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28b5de-17ba-4c1d-8f02-46bf50839e9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:30.784994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9c28b5de-17ba-4c1d-8f02-46bf50839e9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:30.795377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:30.846378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-14ca61c5-5f45-4c30-bfdb-174cbd34ae58 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:30.847596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-14ca61c5-5f45-4c30-bfdb-174cbd34ae58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:30.876945 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28b5de-17ba-4c1d-8f02-46bf50839e9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:30.878573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28b5de-17ba-4c1d-8f02-46bf50839e9e service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:30.898137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 696/1392] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:30.954147 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 697/1393] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:44:31.250712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1080e-af89-42aa-a7fe-f80a0f987107 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:31.251180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1080e-af89-42aa-a7fe-f80a0f987107 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:31.251406 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1080e-af89-42aa-a7fe-f80a0f987107 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:31.252853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 697/1394] 10.222.0.36 () {54 vars in 967 bytes} [Thu Aug 18 14:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:31.258789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de849cc-5116-4260-a5d9-5ed202437baf None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:31.259203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de849cc-5116-4260-a5d9-5ed202437baf None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:31.259451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de849cc-5116-4260-a5d9-5ed202437baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:31.334301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-0de849cc-5116-4260-a5d9-5ed202437baf None None] MFA Rules not processed for user `5f2b6dc3406f46e68c74677f5fc7e3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:31.335505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0de849cc-5116-4260-a5d9-5ed202437baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:31.370520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:31.412494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 698/1395] 10.222.0.36 () {58 vars in 1064 bytes} [Thu Aug 18 14:44:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:31.419939 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1080e-af89-42aa-a7fe-f80a0f987107 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:31.424954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6ce1080e-af89-42aa-a7fe-f80a0f987107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:31.448240 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:31.528907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1080e-af89-42aa-a7fe-f80a0f987107 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:31.600429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406603b2-fa54-4f28-85c2-1be6fe1ae9cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f2b6dc3406f46e68c74677f5fc7e3b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:31.601030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406603b2-fa54-4f28-85c2-1be6fe1ae9cb service glance] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:31.601303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406603b2-fa54-4f28-85c2-1be6fe1ae9cb service glance] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:31.601571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406603b2-fa54-4f28-85c2-1be6fe1ae9cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:31.603183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-406603b2-fa54-4f28-85c2-1be6fe1ae9cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:31.675629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:31.712328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406603b2-fa54-4f28-85c2-1be6fe1ae9cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:31.713777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406603b2-fa54-4f28-85c2-1be6fe1ae9cb service glance] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:31.732803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 698/1396] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:32.543005 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb45be-4b0a-414b-ab8b-97a9df80d450 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:32.543565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb45be-4b0a-414b-ab8b-97a9df80d450 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:32.543565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb45be-4b0a-414b-ab8b-97a9df80d450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:32.610753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-e5fb45be-4b0a-414b-ab8b-97a9df80d450 None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:32.611992 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e5fb45be-4b0a-414b-ab8b-97a9df80d450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:32.702100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 699/1397] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:44:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:32.751402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916f3f5-a006-45f5-bd91-f4d715fd2d6d None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:32.751921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916f3f5-a006-45f5-bd91-f4d715fd2d6d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:32.752111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916f3f5-a006-45f5-bd91-f4d715fd2d6d None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:32.753529 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 699/1398] 10.222.0.36 () {54 vars in 943 bytes} [Thu Aug 18 14:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:32.760209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb474b-d7b1-492b-b170-49bb3cc0e725 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:32.760486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb474b-d7b1-492b-b170-49bb3cc0e725 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:32.760754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb474b-d7b1-492b-b170-49bb3cc0e725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:32.819590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:32.846253 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-9bdb474b-d7b1-492b-b170-49bb3cc0e725 None None] MFA Rules not processed for user `f578727b96564c3db6b5a1e6cac109d8`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:32.847703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9bdb474b-d7b1-492b-b170-49bb3cc0e725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:32.970767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 700/1399] 10.222.0.36 () {58 vars in 1040 bytes} [Thu Aug 18 14:44:32 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:44:32.976686 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f828a33-e3fc-46fc-a997-ae5109bca000 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:32.977569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f828a33-e3fc-46fc-a997-ae5109bca000 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:32.980779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f828a33-e3fc-46fc-a997-ae5109bca000 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:32.980779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 700/1400] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:32.982652 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb474b-d7b1-492b-b170-49bb3cc0e725 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:32.991422 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8c72-a56d-468a-8d89-53da3ab3e5bc None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:32.991619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8c72-a56d-468a-8d89-53da3ab3e5bc None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:32.991933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8c72-a56d-468a-8d89-53da3ab3e5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:32.993360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb474b-d7b1-492b-b170-49bb3cc0e725 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:33.063902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-e3ce8c72-a56d-468a-8d89-53da3ab3e5bc None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:33.065171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e3ce8c72-a56d-468a-8d89-53da3ab3e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:33.104318 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:33.111094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f63542-aacc-43c6-a8ae-88122bdaf029 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:33.111955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f63542-aacc-43c6-a8ae-88122bdaf029 service swift] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:33.112504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f63542-aacc-43c6-a8ae-88122bdaf029 service swift] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:33.112781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f63542-aacc-43c6-a8ae-88122bdaf029 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:33.114531 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-66f63542-aacc-43c6-a8ae-88122bdaf029 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:33.169093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 701/1401] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:44:33.206423 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f63542-aacc-43c6-a8ae-88122bdaf029 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:33.208119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f63542-aacc-43c6-a8ae-88122bdaf029 service swift] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:33.219415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 701/1402] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:44:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:44:33.469486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce8c72-a56d-468a-8d89-53da3ab3e5bc None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:33.470830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e3ce8c72-a56d-468a-8d89-53da3ab3e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:33.512270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:33.620958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce8c72-a56d-468a-8d89-53da3ab3e5bc None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:33.777992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf8109-51ec-48fd-8eea-8d9db329e5f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:33.779834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf8109-51ec-48fd-8eea-8d9db329e5f7 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:33.780115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf8109-51ec-48fd-8eea-8d9db329e5f7 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:33.780389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf8109-51ec-48fd-8eea-8d9db329e5f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:33.782610 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4adf8109-51ec-48fd-8eea-8d9db329e5f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:33.818926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:33.894656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf8109-51ec-48fd-8eea-8d9db329e5f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:33.896593 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf8109-51ec-48fd-8eea-8d9db329e5f7 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:33.918995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 702/1403] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:35.895865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207d51e-f212-4a06-b8be-307084e891a6 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:35.896424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207d51e-f212-4a06-b8be-307084e891a6 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:35.896797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207d51e-f212-4a06-b8be-307084e891a6 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:35.898218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 702/1404] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:35.903771 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860831e-1f5e-4c30-a134-2f6e5892f4e5 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:35.904231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860831e-1f5e-4c30-a134-2f6e5892f4e5 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:35.904336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860831e-1f5e-4c30-a134-2f6e5892f4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:35.980834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-d860831e-1f5e-4c30-a134-2f6e5892f4e5 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:35.982188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d860831e-1f5e-4c30-a134-2f6e5892f4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:36.105696 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 703/1405] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:36.408324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207d51e-f212-4a06-b8be-307084e891a6 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:36.409963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4207d51e-f212-4a06-b8be-307084e891a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:36.587402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:36.600363 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207d51e-f212-4a06-b8be-307084e891a6 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:36.733487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85ea66-9ac9-4188-8896-32d6fbc60472 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:36.734101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85ea66-9ac9-4188-8896-32d6fbc60472 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:36.734378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85ea66-9ac9-4188-8896-32d6fbc60472 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:36.734645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85ea66-9ac9-4188-8896-32d6fbc60472 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:36.736401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ff85ea66-9ac9-4188-8896-32d6fbc60472 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:36.843720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85ea66-9ac9-4188-8896-32d6fbc60472 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:36.845573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85ea66-9ac9-4188-8896-32d6fbc60472 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:36.863310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:36.874049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 703/1406] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:38.943728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ad86d-5997-4ac4-a106-60cf08909b40 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:38.943728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ad86d-5997-4ac4-a106-60cf08909b40 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:38.943728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ad86d-5997-4ac4-a106-60cf08909b40 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:38.943728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 704/1407] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:38.950435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87055cec-4bc5-4b7a-b5bc-6626b7d7247b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:38.950928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87055cec-4bc5-4b7a-b5bc-6626b7d7247b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:38.951064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87055cec-4bc5-4b7a-b5bc-6626b7d7247b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:39.031208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-87055cec-4bc5-4b7a-b5bc-6626b7d7247b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:39.032480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-87055cec-4bc5-4b7a-b5bc-6626b7d7247b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:39.143847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 704/1408] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:39.431073 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ad86d-5997-4ac4-a106-60cf08909b40 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:39.432657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-720ad86d-5997-4ac4-a106-60cf08909b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:39.437941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:39.661687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ad86d-5997-4ac4-a106-60cf08909b40 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:39.755552 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:39.805251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3997fb-32a9-4d0d-af81-3dabf4c19ad2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:39.806166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3997fb-32a9-4d0d-af81-3dabf4c19ad2 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:39.806448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3997fb-32a9-4d0d-af81-3dabf4c19ad2 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:39.806719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3997fb-32a9-4d0d-af81-3dabf4c19ad2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:39.808345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4f3997fb-32a9-4d0d-af81-3dabf4c19ad2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:39.931995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3997fb-32a9-4d0d-af81-3dabf4c19ad2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:39.933803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3997fb-32a9-4d0d-af81-3dabf4c19ad2 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:39.954486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 705/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:44:49.138706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=67 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:49.951854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=61 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:55.989610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fb10c-76a5-48ec-8640-f19fb765bc32 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:55.990118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fb10c-76a5-48ec-8640-f19fb765bc32 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:55.990232 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fb10c-76a5-48ec-8640-f19fb765bc32 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:55.992049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 705/1410] 10.222.0.36 () {54 vars in 918 bytes} [Thu Aug 18 14:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:55.997349 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c0fa2-b21b-496c-9e79-53e51b69724f None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:55.997813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c0fa2-b21b-496c-9e79-53e51b69724f None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:55.997923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c0fa2-b21b-496c-9e79-53e51b69724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:56.191984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-973c0fa2-b21b-496c-9e79-53e51b69724f None None] MFA Rules not processed for user `db581b10d78a4f9eae33fadce0d92b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:56.193944 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-973c0fa2-b21b-496c-9e79-53e51b69724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:56.297268 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 706/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Aug 18 14:44:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:56.312429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f455e8-743a-4cb0-b54c-8b00cf0f4960 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:56.312712 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f455e8-743a-4cb0-b54c-8b00cf0f4960 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:56.312981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f455e8-743a-4cb0-b54c-8b00cf0f4960 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:56.314306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 706/1412] 10.222.0.36 () {54 vars in 918 bytes} [Thu Aug 18 14:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:56.318899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c906656-6199-424f-a8ad-2d8ad6b34d22 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:56.319181 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c906656-6199-424f-a8ad-2d8ad6b34d22 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:56.319484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c906656-6199-424f-a8ad-2d8ad6b34d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:56.392231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:56.395248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-7c906656-6199-424f-a8ad-2d8ad6b34d22 None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:56.396886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7c906656-6199-424f-a8ad-2d8ad6b34d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:56.511917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 707/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Aug 18 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:44:58.903144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c5fd-91f8-438e-a2f5-bdb7902c7c72 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:58.903655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c5fd-91f8-438e-a2f5-bdb7902c7c72 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:58.903777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c5fd-91f8-438e-a2f5-bdb7902c7c72 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:58.905334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:44:58.910640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcc261-488b-4bba-a705-a18bf26051ff None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:58.911118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcc261-488b-4bba-a705-a18bf26051ff None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:58.911215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcc261-488b-4bba-a705-a18bf26051ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:58.979557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-bfbcc261-488b-4bba-a705-a18bf26051ff None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:44:58.980818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bfbcc261-488b-4bba-a705-a18bf26051ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:59.034966 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:59.078388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:44:59.364811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164c5fd-91f8-438e-a2f5-bdb7902c7c72 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:44:59.366495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5164c5fd-91f8-438e-a2f5-bdb7902c7c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:59.513760 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164c5fd-91f8-438e-a2f5-bdb7902c7c72 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:44:59.589786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:44:59.631125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853d829-4951-4ff1-8161-44bc60d8215e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:44:59.631650 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853d829-4951-4ff1-8161-44bc60d8215e service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:44:59.632463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853d829-4951-4ff1-8161-44bc60d8215e service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:44:59.632734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853d829-4951-4ff1-8161-44bc60d8215e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:44:59.634403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f853d829-4951-4ff1-8161-44bc60d8215e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:44:59.726671 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853d829-4951-4ff1-8161-44bc60d8215e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:44:59.728351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853d829-4951-4ff1-8161-44bc60d8215e service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:44:59.747389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 708/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:44:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:00.648114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a245475-7b54-4e64-b6b2-57d2b2d9a9c2 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:00.648655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a245475-7b54-4e64-b6b2-57d2b2d9a9c2 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:00.648655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a245475-7b54-4e64-b6b2-57d2b2d9a9c2 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:00.649983 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 14:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:00.657064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d1990-24f6-4ec7-abc7-830a9cf3f85e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:00.657549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d1990-24f6-4ec7-abc7-830a9cf3f85e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:00.657655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d1990-24f6-4ec7-abc7-830a9cf3f85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:00.736878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-072d1990-24f6-4ec7-abc7-830a9cf3f85e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:00.738096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-072d1990-24f6-4ec7-abc7-830a9cf3f85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:00.766696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:00.837471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:00.987000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a245475-7b54-4e64-b6b2-57d2b2d9a9c2 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:00.988543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1a245475-7b54-4e64-b6b2-57d2b2d9a9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:01.131117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a245475-7b54-4e64-b6b2-57d2b2d9a9c2 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:01.172614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:01.243793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b51c1-ebc7-4201-9117-c8991ef6bdd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:01.244403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b51c1-ebc7-4201-9117-c8991ef6bdd3 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:01.244677 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b51c1-ebc7-4201-9117-c8991ef6bdd3 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:01.244942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b51c1-ebc7-4201-9117-c8991ef6bdd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:01.246547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-af4b51c1-ebc7-4201-9117-c8991ef6bdd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:01.333195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b51c1-ebc7-4201-9117-c8991ef6bdd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:01.334787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b51c1-ebc7-4201-9117-c8991ef6bdd3 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:01.353408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 710/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:02.244889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6e6a4-e927-4d0d-a7b9-179f7b33e3ab None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:02.245495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6e6a4-e927-4d0d-a7b9-179f7b33e3ab None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:02.245594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6e6a4-e927-4d0d-a7b9-179f7b33e3ab None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:02.246878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 14:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:02.252007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea6ffc-8d38-4cdf-a22f-0c77c7842976 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:02.252470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea6ffc-8d38-4cdf-a22f-0c77c7842976 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:02.252778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea6ffc-8d38-4cdf-a22f-0c77c7842976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:02.306805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:02.328834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-b0ea6ffc-8d38-4cdf-a22f-0c77c7842976 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:02.330033 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b0ea6ffc-8d38-4cdf-a22f-0c77c7842976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:02.425194 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 14:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:45:02.578248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6e6a4-e927-4d0d-a7b9-179f7b33e3ab None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:02.580108 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-bdf6e6a4-e927-4d0d-a7b9-179f7b33e3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:02.728135 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6e6a4-e927-4d0d-a7b9-179f7b33e3ab None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:02.740557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:02.841396 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507f821-7586-4fbe-8a4a-b5f80f510fa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:02.841912 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507f821-7586-4fbe-8a4a-b5f80f510fa3 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:02.842186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507f821-7586-4fbe-8a4a-b5f80f510fa3 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:02.842453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507f821-7586-4fbe-8a4a-b5f80f510fa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:02.844076 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8507f821-7586-4fbe-8a4a-b5f80f510fa3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:02.937672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507f821-7586-4fbe-8a4a-b5f80f510fa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:02.939305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507f821-7586-4fbe-8a4a-b5f80f510fa3 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:02.958338 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 711/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:03.821011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9112e-cafe-4f3b-a748-615b7f07d231 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:03.821566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9112e-cafe-4f3b-a748-615b7f07d231 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:03.821566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9112e-cafe-4f3b-a748-615b7f07d231 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:03.822879 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 14:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:03.827842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f9e7e-d12d-45ab-af59-a728a4b063cf None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:03.828283 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f9e7e-d12d-45ab-af59-a728a4b063cf None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:03.828393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f9e7e-d12d-45ab-af59-a728a4b063cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:03.848632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:03.908583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-855f9e7e-d12d-45ab-af59-a728a4b063cf None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:03.909795 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-855f9e7e-d12d-45ab-af59-a728a4b063cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:04.009953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 14:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:04.160708 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9112e-cafe-4f3b-a748-615b7f07d231 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:04.162139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-64a9112e-cafe-4f3b-a748-615b7f07d231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:04.272949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:04.309901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9112e-cafe-4f3b-a748-615b7f07d231 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:04.422233 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd1782-04c0-4c05-8b52-4f007e7733cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:04.422752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd1782-04c0-4c05-8b52-4f007e7733cb service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:04.423027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd1782-04c0-4c05-8b52-4f007e7733cb service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:04.423294 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd1782-04c0-4c05-8b52-4f007e7733cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:04.424932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5cbd1782-04c0-4c05-8b52-4f007e7733cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:04.500306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:04.515401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd1782-04c0-4c05-8b52-4f007e7733cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:04.517029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd1782-04c0-4c05-8b52-4f007e7733cb service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:04.534504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 713/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:05.426577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e49104-f6b1-48b4-9c8f-a6761b7c287e None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:05.427128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e49104-f6b1-48b4-9c8f-a6761b7c287e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:05.427128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e49104-f6b1-48b4-9c8f-a6761b7c287e None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:05.429217 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 14:45:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:05.434224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c92107-7e62-430e-bb03-32bde50a42eb None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:05.434653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c92107-7e62-430e-bb03-32bde50a42eb None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:05.434802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c92107-7e62-430e-bb03-32bde50a42eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:05.512758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-69c92107-7e62-430e-bb03-32bde50a42eb None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:05.513977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-69c92107-7e62-430e-bb03-32bde50a42eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:05.609054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 14:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:05.765690 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e49104-f6b1-48b4-9c8f-a6761b7c287e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:05.767166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b0e49104-f6b1-48b4-9c8f-a6761b7c287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:05.859077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:05.928242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e49104-f6b1-48b4-9c8f-a6761b7c287e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:06.045599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3194f4-f542-458e-b8ca-afcb488f850a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:06.046135 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3194f4-f542-458e-b8ca-afcb488f850a service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:06.046391 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3194f4-f542-458e-b8ca-afcb488f850a service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:06.046661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3194f4-f542-458e-b8ca-afcb488f850a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:06.048302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8a3194f4-f542-458e-b8ca-afcb488f850a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:06.101603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:06.149518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3194f4-f542-458e-b8ca-afcb488f850a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:06.151130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3194f4-f542-458e-b8ca-afcb488f850a service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:06.171331 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 714/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:06.985224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692c790-189d-40f6-afcd-4a8ef16d69dc None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:06.985793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692c790-189d-40f6-afcd-4a8ef16d69dc None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:06.985793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692c790-189d-40f6-afcd-4a8ef16d69dc None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:06.987083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 14:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:06.992648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46049d1f-393a-445f-b434-5d5c99676a63 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:06.993087 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46049d1f-393a-445f-b434-5d5c99676a63 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:06.993181 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46049d1f-393a-445f-b434-5d5c99676a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:07.068581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-46049d1f-393a-445f-b434-5d5c99676a63 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:07.069718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-46049d1f-393a-445f-b434-5d5c99676a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:07.175516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 14:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:07.325527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692c790-189d-40f6-afcd-4a8ef16d69dc None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:07.327077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8692c790-189d-40f6-afcd-4a8ef16d69dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:07.377239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:07.474211 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692c790-189d-40f6-afcd-4a8ef16d69dc None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:07.594270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600b4a2-279b-41cb-9b21-448076549173 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:07.594875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600b4a2-279b-41cb-9b21-448076549173 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:07.595147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600b4a2-279b-41cb-9b21-448076549173 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:07.595434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600b4a2-279b-41cb-9b21-448076549173 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:07.599091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c600b4a2-279b-41cb-9b21-448076549173 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:07.608570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:07.692326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600b4a2-279b-41cb-9b21-448076549173 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:07.693944 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600b4a2-279b-41cb-9b21-448076549173 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:07.712650 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 716/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:08.528837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41f85d-da0f-429b-85d5-72745dadd822 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:08.529340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41f85d-da0f-429b-85d5-72745dadd822 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:08.529454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41f85d-da0f-429b-85d5-72745dadd822 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:08.530737 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 14:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:08.535666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a6540-b338-45ea-a2d9-e4bd84104ea9 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:08.536106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a6540-b338-45ea-a2d9-e4bd84104ea9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:08.536216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a6540-b338-45ea-a2d9-e4bd84104ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:08.613096 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-684a6540-b338-45ea-a2d9-e4bd84104ea9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:08.614316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-684a6540-b338-45ea-a2d9-e4bd84104ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:08.684882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:08.714192 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 14:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:08.874725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41f85d-da0f-429b-85d5-72745dadd822 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:08.876198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-aa41f85d-da0f-429b-85d5-72745dadd822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:08.895773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:09.026957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41f85d-da0f-429b-85d5-72745dadd822 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:09.117410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:09.144289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645f0c7-ebc7-492b-a3a5-823be0274f51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:09.144823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645f0c7-ebc7-492b-a3a5-823be0274f51 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:09.145096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645f0c7-ebc7-492b-a3a5-823be0274f51 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:09.145361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645f0c7-ebc7-492b-a3a5-823be0274f51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:09.146952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a645f0c7-ebc7-492b-a3a5-823be0274f51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:09.239007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645f0c7-ebc7-492b-a3a5-823be0274f51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:09.240638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645f0c7-ebc7-492b-a3a5-823be0274f51 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:09.260182 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 717/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:10.070695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cff12b-15c7-4e23-af5a-cb7b210ecc14 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:10.071236 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cff12b-15c7-4e23-af5a-cb7b210ecc14 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:10.071236 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cff12b-15c7-4e23-af5a-cb7b210ecc14 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:10.072504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 14:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:10.077387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ebeba-4861-4724-8954-5f7be6f84216 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:10.077821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ebeba-4861-4724-8954-5f7be6f84216 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:10.077925 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ebeba-4861-4724-8954-5f7be6f84216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:10.155116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-679ebeba-4861-4724-8954-5f7be6f84216 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:10.156355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-679ebeba-4861-4724-8954-5f7be6f84216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:10.199261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:10.258595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 14:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:10.409381 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cff12b-15c7-4e23-af5a-cb7b210ecc14 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:10.410827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b2cff12b-15c7-4e23-af5a-cb7b210ecc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:10.555984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cff12b-15c7-4e23-af5a-cb7b210ecc14 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:10.609607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:10.669071 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efd413-8eda-49a4-81dd-a3dfce6f2d26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:10.669598 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efd413-8eda-49a4-81dd-a3dfce6f2d26 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:10.669869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efd413-8eda-49a4-81dd-a3dfce6f2d26 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:10.670152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efd413-8eda-49a4-81dd-a3dfce6f2d26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:10.671749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-13efd413-8eda-49a4-81dd-a3dfce6f2d26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:10.758886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efd413-8eda-49a4-81dd-a3dfce6f2d26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:10.760505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efd413-8eda-49a4-81dd-a3dfce6f2d26 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:10.778176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 719/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:11.572617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8aa07-e3dc-487b-a81d-7a806d67cc5e None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:11.573195 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8aa07-e3dc-487b-a81d-7a806d67cc5e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:11.573195 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8aa07-e3dc-487b-a81d-7a806d67cc5e None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:11.574505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 14:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:11.579735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484aa63c-3334-4b1a-9133-31e67994850f None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:11.580183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484aa63c-3334-4b1a-9133-31e67994850f None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:11.580309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484aa63c-3334-4b1a-9133-31e67994850f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:11.656605 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-484aa63c-3334-4b1a-9133-31e67994850f None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:11.657842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-484aa63c-3334-4b1a-9133-31e67994850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:11.659908 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:11.754189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:11.904481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8aa07-e3dc-487b-a81d-7a806d67cc5e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:11.906028 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c8a8aa07-e3dc-487b-a81d-7a806d67cc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:12.049240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8aa07-e3dc-487b-a81d-7a806d67cc5e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:12.076546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:12.165522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6571da7-07a8-49ef-ac56-eb5b396d1602 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:12.166045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571da7-07a8-49ef-ac56-eb5b396d1602 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:12.166324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571da7-07a8-49ef-ac56-eb5b396d1602 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:12.166597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571da7-07a8-49ef-ac56-eb5b396d1602 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:12.168263 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d6571da7-07a8-49ef-ac56-eb5b396d1602 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:12.261603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571da7-07a8-49ef-ac56-eb5b396d1602 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:12.263217 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571da7-07a8-49ef-ac56-eb5b396d1602 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:12.281396 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 720/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:45:14.102014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f669312-c99d-4e7e-b0ea-d9dfcf6883e4 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:14.102561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f669312-c99d-4e7e-b0ea-d9dfcf6883e4 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:14.102561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f669312-c99d-4e7e-b0ea-d9dfcf6883e4 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:14.103938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:14.110623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad33d3-b703-4a51-857b-4f1815dec2db None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:14.111154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad33d3-b703-4a51-857b-4f1815dec2db None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:14.111598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad33d3-b703-4a51-857b-4f1815dec2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:14.160659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:14.187845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-2aad33d3-b703-4a51-857b-4f1815dec2db None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:14.189065 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2aad33d3-b703-4a51-857b-4f1815dec2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:14.287910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:14.582772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f669312-c99d-4e7e-b0ea-d9dfcf6883e4 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:14.584429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1f669312-c99d-4e7e-b0ea-d9dfcf6883e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:14.706574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f669312-c99d-4e7e-b0ea-d9dfcf6883e4 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:14.740586 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:14.792524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e7474-86f1-47ef-a098-d6bca2ada879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:14.793085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e7474-86f1-47ef-a098-d6bca2ada879 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:14.793360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e7474-86f1-47ef-a098-d6bca2ada879 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:14.793624 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e7474-86f1-47ef-a098-d6bca2ada879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:14.795300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-780e7474-86f1-47ef-a098-d6bca2ada879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:14.920833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e7474-86f1-47ef-a098-d6bca2ada879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:14.922326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e7474-86f1-47ef-a098-d6bca2ada879 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:14.945754 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 722/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:17.094424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d6e51-5d98-4a87-965e-f143542606ab None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:17.095004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d6e51-5d98-4a87-965e-f143542606ab None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:17.095004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d6e51-5d98-4a87-965e-f143542606ab None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:17.096345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:17.101715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08c7aa-620d-46dc-90f0-5c1457c769c5 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:17.102216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08c7aa-620d-46dc-90f0-5c1457c769c5 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:17.102332 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08c7aa-620d-46dc-90f0-5c1457c769c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:17.174807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-8b08c7aa-620d-46dc-90f0-5c1457c769c5 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:17.176347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8b08c7aa-620d-46dc-90f0-5c1457c769c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:17.190647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:17.281611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:17.572926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d6e51-5d98-4a87-965e-f143542606ab None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:17.574487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3c7d6e51-5d98-4a87-965e-f143542606ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:17.688431 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d6e51-5d98-4a87-965e-f143542606ab None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:17.702672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:17.771234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144ff56-2fe7-41e5-a301-3d0c7bdebd03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:17.772038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144ff56-2fe7-41e5-a301-3d0c7bdebd03 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:17.772319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144ff56-2fe7-41e5-a301-3d0c7bdebd03 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:17.772621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144ff56-2fe7-41e5-a301-3d0c7bdebd03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:17.774262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a144ff56-2fe7-41e5-a301-3d0c7bdebd03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:17.893182 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144ff56-2fe7-41e5-a301-3d0c7bdebd03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:17.894661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144ff56-2fe7-41e5-a301-3d0c7bdebd03 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:17.913611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 723/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:20.258602 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cd4ed-3638-4586-a9fb-38306ec90490 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:20.259237 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cd4ed-3638-4586-a9fb-38306ec90490 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:20.259237 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cd4ed-3638-4586-a9fb-38306ec90490 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:20.260489 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:45:20.266132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976171c-ff39-45bb-ab77-8193229b4c36 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:20.266642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976171c-ff39-45bb-ab77-8193229b4c36 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:20.266754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976171c-ff39-45bb-ab77-8193229b4c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:20.320332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:20.343464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8976171c-ff39-45bb-ab77-8193229b4c36 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:20.344744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8976171c-ff39-45bb-ab77-8193229b4c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:20.448024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:20.736938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cd4ed-3638-4586-a9fb-38306ec90490 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:20.738484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-426cd4ed-3638-4586-a9fb-38306ec90490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:20.847761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cd4ed-3638-4586-a9fb-38306ec90490 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:20.890382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:20.929265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aa49d-5cfc-4f59-9bd4-0147b46991ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:20.929889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aa49d-5cfc-4f59-9bd4-0147b46991ac service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:20.930168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aa49d-5cfc-4f59-9bd4-0147b46991ac service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:20.930441 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aa49d-5cfc-4f59-9bd4-0147b46991ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:20.932140 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2d3aa49d-5cfc-4f59-9bd4-0147b46991ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:21.051122 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aa49d-5cfc-4f59-9bd4-0147b46991ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:21.052587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aa49d-5cfc-4f59-9bd4-0147b46991ac service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:21.072493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 725/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:23.207021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ce004-18de-43ee-bf34-cb3b73145d3c None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:23.207620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ce004-18de-43ee-bf34-cb3b73145d3c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:23.207620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ce004-18de-43ee-bf34-cb3b73145d3c None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:23.208969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:23.214226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa996d26-bdce-41a5-ba20-5d99c0c2efce None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:23.214665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa996d26-bdce-41a5-ba20-5d99c0c2efce None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:23.214796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa996d26-bdce-41a5-ba20-5d99c0c2efce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:23.283665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-aa996d26-bdce-41a5-ba20-5d99c0c2efce None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:23.284917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-aa996d26-bdce-41a5-ba20-5d99c0c2efce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:23.306829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:23.411016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:23.700011 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ce004-18de-43ee-bf34-cb3b73145d3c None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:23.701593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-285ce004-18de-43ee-bf34-cb3b73145d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:23.813501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ce004-18de-43ee-bf34-cb3b73145d3c None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:23.834454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:23.891190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6efb0f-2d72-43b3-bd65-d2e3bf909b8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:23.891840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6efb0f-2d72-43b3-bd65-d2e3bf909b8b service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:23.892124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6efb0f-2d72-43b3-bd65-d2e3bf909b8b service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:23.892394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6efb0f-2d72-43b3-bd65-d2e3bf909b8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:23.894051 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-fe6efb0f-2d72-43b3-bd65-d2e3bf909b8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:24.017314 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6efb0f-2d72-43b3-bd65-d2e3bf909b8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:24.018673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6efb0f-2d72-43b3-bd65-d2e3bf909b8b service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:24.038808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 726/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:45:26.027161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced2a03-f5ea-4461-9d9c-dfeab3352638 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:26.027848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced2a03-f5ea-4461-9d9c-dfeab3352638 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:26.027848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced2a03-f5ea-4461-9d9c-dfeab3352638 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:26.029217 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:45:26.035199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69828789-fa6b-4b81-b98c-510ed4d8c3ca None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:26.035645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69828789-fa6b-4b81-b98c-510ed4d8c3ca None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:26.035789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69828789-fa6b-4b81-b98c-510ed4d8c3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:26.120503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:26.124230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-69828789-fa6b-4b81-b98c-510ed4d8c3ca None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:26.126080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-69828789-fa6b-4b81-b98c-510ed4d8c3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:26.239655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:26.527987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced2a03-f5ea-4461-9d9c-dfeab3352638 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:26.529545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bced2a03-f5ea-4461-9d9c-dfeab3352638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:26.638303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced2a03-f5ea-4461-9d9c-dfeab3352638 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:26.692201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:26.719951 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce94fe-2bc9-4195-b802-ad373faa13ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:26.720568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce94fe-2bc9-4195-b802-ad373faa13ca service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:26.720843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce94fe-2bc9-4195-b802-ad373faa13ca service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:26.721113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce94fe-2bc9-4195-b802-ad373faa13ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:26.722746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b3ce94fe-2bc9-4195-b802-ad373faa13ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:26.840016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce94fe-2bc9-4195-b802-ad373faa13ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:26.841566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce94fe-2bc9-4195-b802-ad373faa13ca service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:26.862710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 728/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:29.259905 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ba359-4d26-484c-9960-7ed2c7deb79d None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:29.260486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ba359-4d26-484c-9960-7ed2c7deb79d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:29.260597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ba359-4d26-484c-9960-7ed2c7deb79d None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:29.262584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:29.268141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38c2e1-9c09-4b31-b290-f941a59bee4d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:29.268605 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38c2e1-9c09-4b31-b290-f941a59bee4d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:29.268721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38c2e1-9c09-4b31-b290-f941a59bee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:29.346480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-6c38c2e1-9c09-4b31-b290-f941a59bee4d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:29.347946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6c38c2e1-9c09-4b31-b290-f941a59bee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:29.381121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:29.452366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:29.733309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ba359-4d26-484c-9960-7ed2c7deb79d None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:29.734858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c53ba359-4d26-484c-9960-7ed2c7deb79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:29.838580 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ba359-4d26-484c-9960-7ed2c7deb79d None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:29.866940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:29.914182 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94209f-1b9e-4c75-ad33-66d6c361e01a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:29.914804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94209f-1b9e-4c75-ad33-66d6c361e01a service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:29.915070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94209f-1b9e-4c75-ad33-66d6c361e01a service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:29.915336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94209f-1b9e-4c75-ad33-66d6c361e01a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:29.917002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-fe94209f-1b9e-4c75-ad33-66d6c361e01a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:30.032351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94209f-1b9e-4c75-ad33-66d6c361e01a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:30.033704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94209f-1b9e-4c75-ad33-66d6c361e01a service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:30.053469 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 729/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:32.038924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf8569-5959-474d-9a0d-0dfd37497b83 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:32.039491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf8569-5959-474d-9a0d-0dfd37497b83 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:32.039491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf8569-5959-474d-9a0d-0dfd37497b83 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:32.040830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:32.046834 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2405a8f-ddaf-4eca-9fe7-d515caff1563 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:32.047267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2405a8f-ddaf-4eca-9fe7-d515caff1563 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:32.047645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2405a8f-ddaf-4eca-9fe7-d515caff1563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:32.134797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-e2405a8f-ddaf-4eca-9fe7-d515caff1563 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:32.136776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e2405a8f-ddaf-4eca-9fe7-d515caff1563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:32.147448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:32.251040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:32.538535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf8569-5959-474d-9a0d-0dfd37497b83 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:32.540178 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-69bf8569-5959-474d-9a0d-0dfd37497b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:32.645594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf8569-5959-474d-9a0d-0dfd37497b83 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:32.702778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:32.721955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605dd0ef-7dba-4736-acd6-52f240e8f0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:32.722563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605dd0ef-7dba-4736-acd6-52f240e8f0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:32.722838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605dd0ef-7dba-4736-acd6-52f240e8f0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:32.723108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605dd0ef-7dba-4736-acd6-52f240e8f0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:32.725334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-605dd0ef-7dba-4736-acd6-52f240e8f0d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:32.840725 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605dd0ef-7dba-4736-acd6-52f240e8f0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:32.842082 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605dd0ef-7dba-4736-acd6-52f240e8f0d2 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:32.861683 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 731/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:34.947552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dd5ee-1323-4ee3-ae55-a4c43bb4208b None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:34.948184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dd5ee-1323-4ee3-ae55-a4c43bb4208b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:34.948184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dd5ee-1323-4ee3-ae55-a4c43bb4208b None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:34.949505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:34.955305 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a916cd-4120-4244-b3f6-53f6191a989d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:34.955806 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a916cd-4120-4244-b3f6-53f6191a989d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:34.955952 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a916cd-4120-4244-b3f6-53f6191a989d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:35.031502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-b7a916cd-4120-4244-b3f6-53f6191a989d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:35.032770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b7a916cd-4120-4244-b3f6-53f6191a989d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:35.072227 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:35.137774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:35.421669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387dd5ee-1323-4ee3-ae55-a4c43bb4208b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:35.423257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-387dd5ee-1323-4ee3-ae55-a4c43bb4208b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:35.528027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387dd5ee-1323-4ee3-ae55-a4c43bb4208b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:35.566832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:35.606643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d7679-4f1e-46aa-a551-72774af77f08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:35.607301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d7679-4f1e-46aa-a551-72774af77f08 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:35.607607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d7679-4f1e-46aa-a551-72774af77f08 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:35.607946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d7679-4f1e-46aa-a551-72774af77f08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:35.609724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0a6d7679-4f1e-46aa-a551-72774af77f08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:35.728148 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d7679-4f1e-46aa-a551-72774af77f08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:35.729618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d7679-4f1e-46aa-a551-72774af77f08 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:35.747740 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 732/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:37.850667 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b2659-3d6b-4f7f-b073-425fd353528e None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:37.851240 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b2659-3d6b-4f7f-b073-425fd353528e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:37.851240 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b2659-3d6b-4f7f-b073-425fd353528e None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:37.852577 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:45:37.858289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76419c44-ed0c-4fa3-9026-50fabb2eec85 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:37.858709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76419c44-ed0c-4fa3-9026-50fabb2eec85 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:37.859067 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76419c44-ed0c-4fa3-9026-50fabb2eec85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:37.941373 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-76419c44-ed0c-4fa3-9026-50fabb2eec85 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:37.942629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-76419c44-ed0c-4fa3-9026-50fabb2eec85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:37.962204 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:38.054638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:38.364324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b2659-3d6b-4f7f-b073-425fd353528e None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:38.365895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5c4b2659-3d6b-4f7f-b073-425fd353528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:38.481986 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b2659-3d6b-4f7f-b073-425fd353528e None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:38.553559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:38.562431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4532f6a-fc24-4862-ba27-62a4fd84828f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:38.563220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4532f6a-fc24-4862-ba27-62a4fd84828f service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:38.563562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4532f6a-fc24-4862-ba27-62a4fd84828f service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:38.563937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4532f6a-fc24-4862-ba27-62a4fd84828f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:38.566043 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b4532f6a-fc24-4862-ba27-62a4fd84828f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:38.688583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4532f6a-fc24-4862-ba27-62a4fd84828f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:38.690028 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4532f6a-fc24-4862-ba27-62a4fd84828f service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:38.712927 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 734/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:45:40.707688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ec553-683c-481e-a789-be795d2cbec0 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:40.708310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ec553-683c-481e-a789-be795d2cbec0 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:40.708310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ec553-683c-481e-a789-be795d2cbec0 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:40.710311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:45:40.717597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2951bd-7aef-40b8-a7d8-98cf42c940f6 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:40.718148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2951bd-7aef-40b8-a7d8-98cf42c940f6 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:40.718254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2951bd-7aef-40b8-a7d8-98cf42c940f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:40.795237 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-9f2951bd-7aef-40b8-a7d8-98cf42c940f6 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:45:40.796457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9f2951bd-7aef-40b8-a7d8-98cf42c940f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:40.852847 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:40.906878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:45:41.219248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ec553-683c-481e-a789-be795d2cbec0 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:45:41.220855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f20ec553-683c-481e-a789-be795d2cbec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:41.336972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ec553-683c-481e-a789-be795d2cbec0 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:45:41.389374 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:41.419824 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f65bb-4f96-465f-aaa5-e58a3a97a7b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:45:41.420450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f65bb-4f96-465f-aaa5-e58a3a97a7b5 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:45:41.420726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f65bb-4f96-465f-aaa5-e58a3a97a7b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:45:41.420996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f65bb-4f96-465f-aaa5-e58a3a97a7b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:45:41.422703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-456f65bb-4f96-465f-aaa5-e58a3a97a7b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:45:41.563507 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f65bb-4f96-465f-aaa5-e58a3a97a7b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:45:41.565202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f65bb-4f96-465f-aaa5-e58a3a97a7b5 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:45:41.587743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 735/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:45:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:45:50.904686 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=22 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:45:51.585854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=66 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:01.571364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a72dc2-6075-4427-a47a-42df75b18c2e None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:01.571973 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a72dc2-6075-4427-a47a-42df75b18c2e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:01.571973 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a72dc2-6075-4427-a47a-42df75b18c2e None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:01.573258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 736/1471] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:46:01.578531 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2851b8-47fb-47bf-b290-27b4d89ccad7 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:01.579004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2851b8-47fb-47bf-b290-27b4d89ccad7 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:01.579110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2851b8-47fb-47bf-b290-27b4d89ccad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:01.648632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-fd2851b8-47fb-47bf-b290-27b4d89ccad7 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:46:01.649855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-fd2851b8-47fb-47bf-b290-27b4d89ccad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:01.755733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 736/1472] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:46:02.056463 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4e3eb-29c4-4056-981a-6b0e6f2e57bf None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:02.056744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4e3eb-29c4-4056-981a-6b0e6f2e57bf None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:02.057015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4e3eb-29c4-4056-981a-6b0e6f2e57bf None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:02.058336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 737/1473] 10.222.0.36 () {54 vars in 967 bytes} [Thu Aug 18 14:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:46:02.063111 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7be8a6-376a-4961-a137-94f36cdc5608 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:02.063421 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7be8a6-376a-4961-a137-94f36cdc5608 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:02.063731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7be8a6-376a-4961-a137-94f36cdc5608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:02.140830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-de7be8a6-376a-4961-a137-94f36cdc5608 None None] MFA Rules not processed for user `5f2b6dc3406f46e68c74677f5fc7e3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:46:02.142049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-de7be8a6-376a-4961-a137-94f36cdc5608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:02.163736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:02.232851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 737/1474] 10.222.0.36 () {58 vars in 1064 bytes} [Thu Aug 18 14:46:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:46:02.240575 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4e3eb-29c4-4056-981a-6b0e6f2e57bf None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:46:02.242127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e2b4e3eb-29c4-4056-981a-6b0e6f2e57bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:02.348914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4e3eb-29c4-4056-981a-6b0e6f2e57bf None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:46:02.429341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bf122-03c2-4ec0-b5f8-65a74b27ab75 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f2b6dc3406f46e68c74677f5fc7e3b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:46:02.429952 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bf122-03c2-4ec0-b5f8-65a74b27ab75 service glance] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:02.430227 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bf122-03c2-4ec0-b5f8-65a74b27ab75 service glance] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:02.430491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bf122-03c2-4ec0-b5f8-65a74b27ab75 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:02.432117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-679bf122-03c2-4ec0-b5f8-65a74b27ab75 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:02.486112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:02.543154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bf122-03c2-4ec0-b5f8-65a74b27ab75 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:46:02.544645 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bf122-03c2-4ec0-b5f8-65a74b27ab75 service glance] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:46:02.563929 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 738/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:46:04.594093 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551bd34-34e1-400b-bba2-43b8e6d1cfc8 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:04.594668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551bd34-34e1-400b-bba2-43b8e6d1cfc8 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:04.594668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551bd34-34e1-400b-bba2-43b8e6d1cfc8 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:04.596051 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 738/1476] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:46:04.601912 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bca4e9-4e8a-4418-981a-45c88cee8ec8 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:04.602415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bca4e9-4e8a-4418-981a-45c88cee8ec8 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:04.602530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bca4e9-4e8a-4418-981a-45c88cee8ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:04.667645 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a7bca4e9-4e8a-4418-981a-45c88cee8ec8 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:46:04.668896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a7bca4e9-4e8a-4418-981a-45c88cee8ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:04.768179 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 739/1477] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:46:05.039155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551bd34-34e1-400b-bba2-43b8e6d1cfc8 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:46:05.040777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f551bd34-34e1-400b-bba2-43b8e6d1cfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:05.173328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551bd34-34e1-400b-bba2-43b8e6d1cfc8 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:46:05.223749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:05.277389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e56ce-07a9-495f-b0d7-150c282f8330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:46:05.278007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e56ce-07a9-495f-b0d7-150c282f8330 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:05.279946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e56ce-07a9-495f-b0d7-150c282f8330 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:05.280594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e56ce-07a9-495f-b0d7-150c282f8330 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:05.282046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e56ce-07a9-495f-b0d7-150c282f8330 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:46:05.285390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: WARNING py.warnings [None req-cd8e56ce-07a9-495f-b0d7-150c282f8330 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:46:05.285390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: warnings.warn(msg) Aug 18 14:46:05.285390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: Aug 18 14:46:05.286557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e56ce-07a9-495f-b0d7-150c282f8330 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:46:05.306778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 739/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Aug 18 14:46:05 2022] GET /identity/v3/projects => generated 2553 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:46:07.439131 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5aea3e-298f-4198-920e-0c2dfde94137 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:07.439784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5aea3e-298f-4198-920e-0c2dfde94137 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:07.439784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5aea3e-298f-4198-920e-0c2dfde94137 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:07.441016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 740/1479] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:46:07.446867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738c908-b7d4-43bf-bac6-109f94d8283d None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:07.447309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738c908-b7d4-43bf-bac6-109f94d8283d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:07.447885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738c908-b7d4-43bf-bac6-109f94d8283d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:07.647574 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4738c908-b7d4-43bf-bac6-109f94d8283d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:46:07.648752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4738c908-b7d4-43bf-bac6-109f94d8283d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:07.748766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 740/1480] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:46:08.092954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5aea3e-298f-4198-920e-0c2dfde94137 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:46:08.094519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5f5aea3e-298f-4198-920e-0c2dfde94137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:08.104936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:08.267772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5aea3e-298f-4198-920e-0c2dfde94137 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:46:08.347854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:08.382338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb0a42-3b35-4f60-a80e-e1d4ddd9cf14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:46:08.382943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb0a42-3b35-4f60-a80e-e1d4ddd9cf14 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:08.383215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb0a42-3b35-4f60-a80e-e1d4ddd9cf14 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:08.383482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb0a42-3b35-4f60-a80e-e1d4ddd9cf14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:08.385156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6edb0a42-3b35-4f60-a80e-e1d4ddd9cf14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:08.478430 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb0a42-3b35-4f60-a80e-e1d4ddd9cf14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:46:08.480107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb0a42-3b35-4f60-a80e-e1d4ddd9cf14 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:46:08.497994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 741/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:46:10.684660 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29901846-ed05-4aa7-8647-5c772ebcb2af None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:10.685284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29901846-ed05-4aa7-8647-5c772ebcb2af None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:10.685284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29901846-ed05-4aa7-8647-5c772ebcb2af None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:10.686581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 741/1482] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:46:10.691830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6780fe-f418-47c9-a622-a900de7bf751 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:10.692246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6780fe-f418-47c9-a622-a900de7bf751 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:10.692388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6780fe-f418-47c9-a622-a900de7bf751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:10.761753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-de6780fe-f418-47c9-a622-a900de7bf751 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:46:10.762981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-de6780fe-f418-47c9-a622-a900de7bf751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:10.810883 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:10.872038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 742/1483] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:46:11.157436 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29901846-ed05-4aa7-8647-5c772ebcb2af None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:46:11.158971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-29901846-ed05-4aa7-8647-5c772ebcb2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:11.182710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:11.262439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29901846-ed05-4aa7-8647-5c772ebcb2af None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:46:11.334580 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d3a73-7e7c-4d38-b253-af5f48adf867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:46:11.335205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d3a73-7e7c-4d38-b253-af5f48adf867 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:11.335484 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d3a73-7e7c-4d38-b253-af5f48adf867 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:11.335777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d3a73-7e7c-4d38-b253-af5f48adf867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:11.337552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d44d3a73-7e7c-4d38-b253-af5f48adf867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:11.415548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:11.451503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d3a73-7e7c-4d38-b253-af5f48adf867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:46:11.452977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d3a73-7e7c-4d38-b253-af5f48adf867 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:46:11.470885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 742/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:46:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:46:13.493665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa18929-9b2d-46ca-be78-4884353bd259 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:13.494238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa18929-9b2d-46ca-be78-4884353bd259 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:13.494238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa18929-9b2d-46ca-be78-4884353bd259 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:13.495558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 743/1485] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:46:13.500806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686a313-35df-4598-a749-03fd25a4ef8f None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:13.501252 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686a313-35df-4598-a749-03fd25a4ef8f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:13.501365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686a313-35df-4598-a749-03fd25a4ef8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:13.566886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-7686a313-35df-4598-a749-03fd25a4ef8f None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:46:13.568116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7686a313-35df-4598-a749-03fd25a4ef8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:13.667223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 743/1486] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:46:13.966444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa18929-9b2d-46ca-be78-4884353bd259 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:46:13.968017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8fa18929-9b2d-46ca-be78-4884353bd259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:14.072404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa18929-9b2d-46ca-be78-4884353bd259 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:46:14.155059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:14.166063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f0e87-1986-41e5-a0de-53e2b2bd11f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:46:14.166662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f0e87-1986-41e5-a0de-53e2b2bd11f4 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:14.166940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f0e87-1986-41e5-a0de-53e2b2bd11f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:14.167202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f0e87-1986-41e5-a0de-53e2b2bd11f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:14.168818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f05f0e87-1986-41e5-a0de-53e2b2bd11f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:14.274336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f0e87-1986-41e5-a0de-53e2b2bd11f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:46:14.275780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f0e87-1986-41e5-a0de-53e2b2bd11f4 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:46:14.293361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 744/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:46:16.418739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356045c-cb5d-46fd-b757-885c0cce204a None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:16.419311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356045c-cb5d-46fd-b757-885c0cce204a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:16.419311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356045c-cb5d-46fd-b757-885c0cce204a None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:16.420646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 744/1488] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:46:16.425953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de94ba-063b-48e0-9135-40db3bef9d98 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:16.426430 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de94ba-063b-48e0-9135-40db3bef9d98 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:16.426530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de94ba-063b-48e0-9135-40db3bef9d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:16.491622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f5de94ba-063b-48e0-9135-40db3bef9d98 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:46:16.492863 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f5de94ba-063b-48e0-9135-40db3bef9d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:16.545828 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:16.591360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 745/1489] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:46:16.906542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356045c-cb5d-46fd-b757-885c0cce204a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:46:16.908159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c356045c-cb5d-46fd-b757-885c0cce204a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:16.944959 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:17.013115 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356045c-cb5d-46fd-b757-885c0cce204a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:46:17.085059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d688d67-c1f3-4dc0-999c-5d1fbcfb0b82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:46:17.085685 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d688d67-c1f3-4dc0-999c-5d1fbcfb0b82 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:17.085987 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d688d67-c1f3-4dc0-999c-5d1fbcfb0b82 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:17.086256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d688d67-c1f3-4dc0-999c-5d1fbcfb0b82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:17.087885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0d688d67-c1f3-4dc0-999c-5d1fbcfb0b82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:17.166565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:17.196347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d688d67-c1f3-4dc0-999c-5d1fbcfb0b82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:46:17.197813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d688d67-c1f3-4dc0-999c-5d1fbcfb0b82 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:46:17.215633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 745/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:46:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:46:19.311902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc13b41-c116-49c4-8be3-dc8c11c39ff8 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:19.312471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc13b41-c116-49c4-8be3-dc8c11c39ff8 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:19.312471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc13b41-c116-49c4-8be3-dc8c11c39ff8 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:19.313786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 746/1491] 10.222.0.36 () {54 vars in 927 bytes} [Thu Aug 18 14:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:46:19.319620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae34dc7-8772-4ea3-84f6-192c1411740e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:19.320097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae34dc7-8772-4ea3-84f6-192c1411740e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:19.320215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae34dc7-8772-4ea3-84f6-192c1411740e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:19.386316 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-aae34dc7-8772-4ea3-84f6-192c1411740e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:46:19.387531 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-aae34dc7-8772-4ea3-84f6-192c1411740e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:19.496642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 746/1492] 10.222.0.36 () {58 vars in 1024 bytes} [Thu Aug 18 14:46:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:46:19.804221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc13b41-c116-49c4-8be3-dc8c11c39ff8 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:46:19.805938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-acc13b41-c116-49c4-8be3-dc8c11c39ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:19.927767 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc13b41-c116-49c4-8be3-dc8c11c39ff8 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:46:20.024899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e90a81-70f6-4393-8f4c-9a23aa831fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:46:20.025911 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e90a81-70f6-4393-8f4c-9a23aa831fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:46:20.026371 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e90a81-70f6-4393-8f4c-9a23aa831fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:46:20.026831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e90a81-70f6-4393-8f4c-9a23aa831fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:46:20.029730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b9e90a81-70f6-4393-8f4c-9a23aa831fb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:46:20.041697 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:20.177097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e90a81-70f6-4393-8f4c-9a23aa831fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:46:20.178588 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e90a81-70f6-4393-8f4c-9a23aa831fb7 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:46:20.197434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 747/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:46:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:46:29.494520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=82 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:46:30.194993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=50 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:47:42.000035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30ce37-70ba-4792-b89d-042a2373c138 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:42.000688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30ce37-70ba-4792-b89d-042a2373c138 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:42.000688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30ce37-70ba-4792-b89d-042a2373c138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:42.090610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-6a30ce37-70ba-4792-b89d-042a2373c138 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:47:42.091986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6a30ce37-70ba-4792-b89d-042a2373c138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:42.196521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 747/1494] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:47:42.205778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17179f7-645b-4495-a70c-be9fda337c7c None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:42.206488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17179f7-645b-4495-a70c-be9fda337c7c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:42.206488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17179f7-645b-4495-a70c-be9fda337c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:42.310510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a17179f7-645b-4495-a70c-be9fda337c7c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:47:42.311788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a17179f7-645b-4495-a70c-be9fda337c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:42.436590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 748/1495] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:47:42.443496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30ce37-70ba-4792-b89d-042a2373c138 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:47:42.444818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6a30ce37-70ba-4792-b89d-042a2373c138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:42.457052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30ce37-70ba-4792-b89d-042a2373c138 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:47:42.530618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:47:42.567698 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc34988-daa4-4da2-92af-8197567d1cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:47:42.568630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc34988-daa4-4da2-92af-8197567d1cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:42.569089 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc34988-daa4-4da2-92af-8197567d1cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:42.569533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc34988-daa4-4da2-92af-8197567d1cb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:42.570578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc34988-daa4-4da2-92af-8197567d1cb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:47:42.571100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc34988-daa4-4da2-92af-8197567d1cb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:47:42.572797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: WARNING py.warnings [None req-acc34988-daa4-4da2-92af-8197567d1cb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:47:42.572797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: warnings.warn(msg) Aug 18 14:47:42.572797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: Aug 18 14:47:42.574674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc34988-daa4-4da2-92af-8197567d1cb8 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:47:42.594333 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 748/1496] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:47:42.603609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17179f7-645b-4495-a70c-be9fda337c7c None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:47:42.613553 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17179f7-645b-4495-a70c-be9fda337c7c None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:47:42.692552 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:47:42.729999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7a88a-00d9-4808-b185-e50e40d0c6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:47:42.730545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7a88a-00d9-4808-b185-e50e40d0c6af admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:42.730821 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7a88a-00d9-4808-b185-e50e40d0c6af admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:42.731090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7a88a-00d9-4808-b185-e50e40d0c6af admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:42.731749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7a88a-00d9-4808-b185-e50e40d0c6af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:47:42.733474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: WARNING py.warnings [None req-16c7a88a-00d9-4808-b185-e50e40d0c6af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:47:42.733474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: warnings.warn(msg) Aug 18 14:47:42.733474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: Aug 18 14:47:42.734665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7a88a-00d9-4808-b185-e50e40d0c6af admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:47:42.794330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 749/1497] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:47:42 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:47:42.799345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc34988-daa4-4da2-92af-8197567d1cb8 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:47:42.801282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-acc34988-daa4-4da2-92af-8197567d1cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:42.845963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc34988-daa4-4da2-92af-8197567d1cb8 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:47:42.971596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2603e-2dbe-4ffe-8f35-e3453d322fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:47:42.972237 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2603e-2dbe-4ffe-8f35-e3453d322fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:42.972532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2603e-2dbe-4ffe-8f35-e3453d322fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:42.972787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2603e-2dbe-4ffe-8f35-e3453d322fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:42.973403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2603e-2dbe-4ffe-8f35-e3453d322fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:47:42.974514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: WARNING py.warnings [None req-7cb2603e-2dbe-4ffe-8f35-e3453d322fe9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:47:42.974514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: warnings.warn(msg) Aug 18 14:47:42.974514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: Aug 18 14:47:42.975979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2603e-2dbe-4ffe-8f35-e3453d322fe9 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:47:43.031841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 749/1498] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:47:42 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:47:43.038546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7a88a-00d9-4808-b185-e50e40d0c6af admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:47:43.040241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-16c7a88a-00d9-4808-b185-e50e40d0c6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:43.109514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7a88a-00d9-4808-b185-e50e40d0c6af admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:47:43.202663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:47:43.215323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015924c-658e-494b-863a-61ebac754385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:47:43.216035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015924c-658e-494b-863a-61ebac754385 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:43.216446 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015924c-658e-494b-863a-61ebac754385 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:43.216744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015924c-658e-494b-863a-61ebac754385 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:43.217474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015924c-658e-494b-863a-61ebac754385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:47:43.218554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: WARNING py.warnings [None req-9015924c-658e-494b-863a-61ebac754385 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:47:43.218554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: warnings.warn(msg) Aug 18 14:47:43.218554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: Aug 18 14:47:43.219433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015924c-658e-494b-863a-61ebac754385 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:47:43.233243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 750/1499] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:47:43.237213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2603e-2dbe-4ffe-8f35-e3453d322fe9 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:47:43.238758 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7cb2603e-2dbe-4ffe-8f35-e3453d322fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:43.250088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2603e-2dbe-4ffe-8f35-e3453d322fe9 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:47:43.282031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:47:43.370006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef98ca-ece8-4543-bae0-eae774996938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:47:43.370609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef98ca-ece8-4543-bae0-eae774996938 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:43.370882 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef98ca-ece8-4543-bae0-eae774996938 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:43.371147 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef98ca-ece8-4543-bae0-eae774996938 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:43.371807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef98ca-ece8-4543-bae0-eae774996938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:47:43.372816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: WARNING py.warnings [None req-58ef98ca-ece8-4543-bae0-eae774996938 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:47:43.372816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: warnings.warn(msg) Aug 18 14:47:43.372816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: Aug 18 14:47:43.373687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef98ca-ece8-4543-bae0-eae774996938 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:47:43.386200 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 750/1500] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:47:43.390582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015924c-658e-494b-863a-61ebac754385 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:47:43.392829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9015924c-658e-494b-863a-61ebac754385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:43.429688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015924c-658e-494b-863a-61ebac754385 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:47:43.534348 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb9355-a7df-4469-81dc-1c376885413e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:47:43.536312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb9355-a7df-4469-81dc-1c376885413e admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:43.536877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb9355-a7df-4469-81dc-1c376885413e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:43.537176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb9355-a7df-4469-81dc-1c376885413e admin admin] PATH_INFO: `/v3/projects/d6eddfccc5434434b489d669fd7727b9/users/b6d1d7d9a8be4150bae13add9a20df65/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:43.561009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb9355-a7df-4469-81dc-1c376885413e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6eddfccc5434434b489d669fd7727b9, user_id=b6d1d7d9a8be4150bae13add9a20df65, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:47:43.562096 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: WARNING py.warnings [None req-8cdb9355-a7df-4469-81dc-1c376885413e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:47:43.562096 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: warnings.warn(msg) Aug 18 14:47:43.562096 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: Aug 18 14:47:43.564298 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb9355-a7df-4469-81dc-1c376885413e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:47:43.591322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 751/1501] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:47:43 2022] PUT /identity/v3/projects/d6eddfccc5434434b489d669fd7727b9/users/b6d1d7d9a8be4150bae13add9a20df65/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:47:43.604113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e95397-f09b-4a0d-91a2-4633e4af30eb None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:43.604394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e95397-f09b-4a0d-91a2-4633e4af30eb None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:43.604663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e95397-f09b-4a0d-91a2-4633e4af30eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:43.667135 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-52e95397-f09b-4a0d-91a2-4633e4af30eb None None] MFA Rules not processed for user `b6d1d7d9a8be4150bae13add9a20df65`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:47:43.668383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-52e95397-f09b-4a0d-91a2-4633e4af30eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:43.753429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 751/1502] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:47:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:47:43.770346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd318d7a-14c1-4e6f-9e96-b30f439c4f7c None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:43.770633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd318d7a-14c1-4e6f-9e96-b30f439c4f7c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:43.770899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd318d7a-14c1-4e6f-9e96-b30f439c4f7c None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:43.772286 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 752/1503] 10.222.0.36 () {54 vars in 964 bytes} [Thu Aug 18 14:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:47:43.777625 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd1619-197a-40fc-9f20-1cef719fe58b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:43.778049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd1619-197a-40fc-9f20-1cef719fe58b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:43.778449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd1619-197a-40fc-9f20-1cef719fe58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:43.815743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:47:43.869244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-eedd1619-197a-40fc-9f20-1cef719fe58b None None] MFA Rules not processed for user `1a9b29546bc848ef9e92b3d267da18ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:47:43.870467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-eedd1619-197a-40fc-9f20-1cef719fe58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:43.954941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 752/1504] 10.222.0.36 () {58 vars in 1061 bytes} [Thu Aug 18 14:47:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:47:43.963355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd318d7a-14c1-4e6f-9e96-b30f439c4f7c None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:47:43.965112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-dd318d7a-14c1-4e6f-9e96-b30f439c4f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:44.020327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:47:44.077698 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd318d7a-14c1-4e6f-9e96-b30f439c4f7c None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:47:44.161896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a50f5-6f9f-4e27-a3bc-981b88a44b6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:47:44.162502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a50f5-6f9f-4e27-a3bc-981b88a44b6b service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:44.162821 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a50f5-6f9f-4e27-a3bc-981b88a44b6b service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:44.163060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a50f5-6f9f-4e27-a3bc-981b88a44b6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:44.164687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a78a50f5-6f9f-4e27-a3bc-981b88a44b6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:44.271391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a50f5-6f9f-4e27-a3bc-981b88a44b6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:47:44.272881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a50f5-6f9f-4e27-a3bc-981b88a44b6b service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:47:44.276574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:47:44.293071 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 753/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:47:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:47:44.347523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76a632-2a47-4b5e-8f0f-302f354a0b6b None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:44.348267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76a632-2a47-4b5e-8f0f-302f354a0b6b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:44.348538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76a632-2a47-4b5e-8f0f-302f354a0b6b None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:44.349892 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 753/1506] 10.222.0.36 () {52 vars in 830 bytes} [Thu Aug 18 14:47:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:47:44.354016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a50f5-6f9f-4e27-a3bc-981b88a44b6b service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:47:44.355194 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a78a50f5-6f9f-4e27-a3bc-981b88a44b6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:44.464864 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a50f5-6f9f-4e27-a3bc-981b88a44b6b service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:47:44.569880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:47:44.570517 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:44.570799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:44.571064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] PATH_INFO: `/v3/users/b6d1d7d9a8be4150bae13add9a20df65` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:44.584405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6d1d7d9a8be4150bae13add9a20df65)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:47:44.585367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: WARNING py.warnings [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:47:44.585367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: warnings.warn(msg) Aug 18 14:47:44.585367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: Aug 18 14:47:44.586626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:47:44.597587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:47:44.661806 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6d1d7d9a8be4150bae13add9a20df65'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:47:44.677388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6d1d7d9a8be4150bae13add9a20df65'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:47:44.677936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6d1d7d9a8be4150bae13add9a20df65'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:47:44.678584 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6d1d7d9a8be4150bae13add9a20df65'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:47:44.707563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6d1d7d9a8be4150bae13add9a20df65'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:47:44.725290 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 754/1507] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:47:44 2022] DELETE /identity/v3/users/b6d1d7d9a8be4150bae13add9a20df65 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:47:44.729848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27ad39-6d6a-4b0f-a881-2d38dcf427dd None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:44.730077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27ad39-6d6a-4b0f-a881-2d38dcf427dd None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:44.730317 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27ad39-6d6a-4b0f-a881-2d38dcf427dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:44.792107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-6b27ad39-6d6a-4b0f-a881-2d38dcf427dd None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:47:44.793336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6b27ad39-6d6a-4b0f-a881-2d38dcf427dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:44.882081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:47:44.897541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 754/1508] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:47:44.908498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:47:44.909756 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:45.020724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a1b16-f62e-4ce3-849a-4eac5f27cdb5 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:47:45.082488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:47:45.108047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582a074-936c-4a57-af2b-48a5cab2f087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:47:45.108680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582a074-936c-4a57-af2b-48a5cab2f087 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:45.108959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582a074-936c-4a57-af2b-48a5cab2f087 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:45.109230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582a074-936c-4a57-af2b-48a5cab2f087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:45.110873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f582a074-936c-4a57-af2b-48a5cab2f087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:45.245355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582a074-936c-4a57-af2b-48a5cab2f087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:47:45.246817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582a074-936c-4a57-af2b-48a5cab2f087 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:47:45.268259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 755/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:47:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:47:45.669871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27ad39-6d6a-4b0f-a881-2d38dcf427dd None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:47:45.671559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6b27ad39-6d6a-4b0f-a881-2d38dcf427dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:47:45.685078 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27ad39-6d6a-4b0f-a881-2d38dcf427dd None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:47:45.797222 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:47:45.798391 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:47:45.798915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:47:45.798915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] PATH_INFO: `/v3/projects/d6eddfccc5434434b489d669fd7727b9` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:47:45.810555 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6eddfccc5434434b489d669fd7727b9)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:47:45.811885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: WARNING py.warnings [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:47:45.811885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: warnings.warn(msg) Aug 18 14:47:45.811885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: Aug 18 14:47:45.814241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:47:45.880028 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6eddfccc5434434b489d669fd7727b9', 'request_id': 'req-c819b918-83fe-4afb-b6b7-e59095a7349d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:47:45.886218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6eddfccc5434434b489d669fd7727b9', 'request_id': 'req-c819b918-83fe-4afb-b6b7-e59095a7349d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:47:45.898463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6eddfccc5434434b489d669fd7727b9', 'request_id': 'req-c819b918-83fe-4afb-b6b7-e59095a7349d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:47:45.934043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] The token cache is being invalidate because project d6eddfccc5434434b489d669fd7727b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:47:45.934364 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:47:45.934861 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c819b918-83fe-4afb-b6b7-e59095a7349d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:47:45.936290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 755/1510] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:47:45 2022] DELETE /identity/v3/projects/d6eddfccc5434434b489d669fd7727b9 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:47:55.265065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=62 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:47:55.931874 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:49:22.030522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132df21-b374-4dbd-a64e-b55a58db1952 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:22.031131 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132df21-b374-4dbd-a64e-b55a58db1952 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:22.031518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132df21-b374-4dbd-a64e-b55a58db1952 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:22.033240 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 14:49:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:49:22.038268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d77007-ac7e-460c-806a-c6cae9f22556 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:22.038717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d77007-ac7e-460c-806a-c6cae9f22556 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:22.038822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d77007-ac7e-460c-806a-c6cae9f22556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:22.122240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-81d77007-ac7e-460c-806a-c6cae9f22556 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:49:22.123627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-81d77007-ac7e-460c-806a-c6cae9f22556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:22.232137 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 14:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:49:22.385161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132df21-b374-4dbd-a64e-b55a58db1952 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:49:22.387390 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-0132df21-b374-4dbd-a64e-b55a58db1952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:22.506312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132df21-b374-4dbd-a64e-b55a58db1952 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:49:22.511839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75431de-71ac-41db-b56e-9af6134d6aba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:49:22.512450 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75431de-71ac-41db-b56e-9af6134d6aba service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:22.512729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75431de-71ac-41db-b56e-9af6134d6aba service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:22.513000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75431de-71ac-41db-b56e-9af6134d6aba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:22.522283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75431de-71ac-41db-b56e-9af6134d6aba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:49:22.523917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75431de-71ac-41db-b56e-9af6134d6aba service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:49:22.537272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 757/1513] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:49:22.937092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be214d11-8ee0-4031-b9be-e22eb05cb69a None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:22.937413 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be214d11-8ee0-4031-b9be-e22eb05cb69a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:22.937638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be214d11-8ee0-4031-b9be-e22eb05cb69a None None] PATH_INFO: `/v3` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:22.938924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 757/1514] 10.222.0.52 () {54 vars in 925 bytes} [Thu Aug 18 14:49:22 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:49:22.946453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42881634-a501-4ceb-8d4f-13b1ac28e4e1 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:22.946852 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42881634-a501-4ceb-8d4f-13b1ac28e4e1 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:22.947295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42881634-a501-4ceb-8d4f-13b1ac28e4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:23.019564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-42881634-a501-4ceb-8d4f-13b1ac28e4e1 None None] MFA Rules not processed for user `db581b10d78a4f9eae33fadce0d92b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:49:23.020864 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-42881634-a501-4ceb-8d4f-13b1ac28e4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:23.093542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:49:23.100415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 758/1515] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Aug 18 14:49:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:49:23.390308 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311ba5c-be82-4a72-8acc-b30387d59fb1 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:23.390845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311ba5c-be82-4a72-8acc-b30387d59fb1 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:23.390845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311ba5c-be82-4a72-8acc-b30387d59fb1 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:23.392249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 758/1516] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 14:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:49:23.397124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cb846-d73a-4c08-9d1e-d0ff98a8115d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:23.397407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cb846-d73a-4c08-9d1e-d0ff98a8115d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:23.397681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cb846-d73a-4c08-9d1e-d0ff98a8115d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:23.417833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-ad1cb846-d73a-4c08-9d1e-d0ff98a8115d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:49:23.419058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ad1cb846-d73a-4c08-9d1e-d0ff98a8115d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:23.427879 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 759/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 14:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:49:23.580799 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311ba5c-be82-4a72-8acc-b30387d59fb1 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:49:23.596454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311ba5c-be82-4a72-8acc-b30387d59fb1 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:49:23.601835 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2d7ff-92f7-4470-942c-a983aa95f3aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:49:23.602431 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2d7ff-92f7-4470-942c-a983aa95f3aa service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:23.602703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2d7ff-92f7-4470-942c-a983aa95f3aa service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:23.602975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2d7ff-92f7-4470-942c-a983aa95f3aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:23.612135 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2d7ff-92f7-4470-942c-a983aa95f3aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:49:23.613733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2d7ff-92f7-4470-942c-a983aa95f3aa service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:49:23.623256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 759/1518] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:49:32.046308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f20e-41de-4567-b385-413f170f0fe9 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:32.046894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f20e-41de-4567-b385-413f170f0fe9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:32.046894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f20e-41de-4567-b385-413f170f0fe9 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:32.048095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 760/1519] 10.222.0.36 () {54 vars in 961 bytes} [Thu Aug 18 14:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:49:32.053551 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d2cb8-53aa-448e-8178-c28ebe8bf83c None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:32.054044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d2cb8-53aa-448e-8178-c28ebe8bf83c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:32.054139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d2cb8-53aa-448e-8178-c28ebe8bf83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:32.073114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-123d2cb8-53aa-448e-8178-c28ebe8bf83c None None] MFA Rules not processed for user `db581b10d78a4f9eae33fadce0d92b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:49:32.074337 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-123d2cb8-53aa-448e-8178-c28ebe8bf83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:32.083784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 760/1520] 10.222.0.36 () {58 vars in 1058 bytes} [Thu Aug 18 14:49:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:49:32.091351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4f20e-41de-4567-b385-413f170f0fe9 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:49:32.105164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4f20e-41de-4567-b385-413f170f0fe9 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:49:32.110079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a3ab9-8384-4fd9-a572-3d3da4ec56d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:49:32.110676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a3ab9-8384-4fd9-a572-3d3da4ec56d1 service placement] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:32.110954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a3ab9-8384-4fd9-a572-3d3da4ec56d1 service placement] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:32.111222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a3ab9-8384-4fd9-a572-3d3da4ec56d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:32.119669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a3ab9-8384-4fd9-a572-3d3da4ec56d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:49:32.121270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a3ab9-8384-4fd9-a572-3d3da4ec56d1 service placement] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:49:32.131349 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 761/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:49:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:49:34.881251 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b800628-a144-48c8-ad8b-2c19ff5de682 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:34.881825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b800628-a144-48c8-ad8b-2c19ff5de682 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:34.882153 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b800628-a144-48c8-ad8b-2c19ff5de682 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:34.884644 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 14:49:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:49:34.890341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5e776-20a6-4aba-b531-b10cc48ab3b4 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:34.890861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5e776-20a6-4aba-b531-b10cc48ab3b4 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:34.890861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5e776-20a6-4aba-b531-b10cc48ab3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:34.910672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-c7f5e776-20a6-4aba-b531-b10cc48ab3b4 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:49:34.911987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c7f5e776-20a6-4aba-b531-b10cc48ab3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:34.920393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 14:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:49:35.076586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b800628-a144-48c8-ad8b-2c19ff5de682 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:49:35.092254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b800628-a144-48c8-ad8b-2c19ff5de682 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:49:35.097588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17e66c-dd04-415a-a119-9139db03c3c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:49:35.098171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17e66c-dd04-415a-a119-9139db03c3c8 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:35.098446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17e66c-dd04-415a-a119-9139db03c3c8 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:35.098713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17e66c-dd04-415a-a119-9139db03c3c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:35.107952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17e66c-dd04-415a-a119-9139db03c3c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:49:35.109541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17e66c-dd04-415a-a119-9139db03c3c8 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:49:35.119530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 762/1524] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:49:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:49:35.968837 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1aefdd-38db-4d1d-bb76-db9ad2fe10b1 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:35.969364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1aefdd-38db-4d1d-bb76-db9ad2fe10b1 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:35.969364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1aefdd-38db-4d1d-bb76-db9ad2fe10b1 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:35.970635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 14:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:49:35.976119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5fe16-bc1e-4e48-84ad-5b3421fe0dff None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:35.976571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5fe16-bc1e-4e48-84ad-5b3421fe0dff None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:35.976810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5fe16-bc1e-4e48-84ad-5b3421fe0dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:35.997522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-73e5fe16-bc1e-4e48-84ad-5b3421fe0dff None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:49:35.999389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-73e5fe16-bc1e-4e48-84ad-5b3421fe0dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:36.008941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 14:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:49:36.162166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1aefdd-38db-4d1d-bb76-db9ad2fe10b1 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:49:36.177158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1aefdd-38db-4d1d-bb76-db9ad2fe10b1 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:49:36.182543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026596ab-55a7-406a-99e6-71e16de1e93f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:49:36.183146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026596ab-55a7-406a-99e6-71e16de1e93f service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:36.183419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026596ab-55a7-406a-99e6-71e16de1e93f service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:36.183713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026596ab-55a7-406a-99e6-71e16de1e93f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:36.192490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026596ab-55a7-406a-99e6-71e16de1e93f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:49:36.194107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026596ab-55a7-406a-99e6-71e16de1e93f service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:49:36.204031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 764/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:49:40.171008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7df50-6d8d-410d-947a-97f5fcd2ebc8 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:40.171546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7df50-6d8d-410d-947a-97f5fcd2ebc8 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:40.171546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7df50-6d8d-410d-947a-97f5fcd2ebc8 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:40.172890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:49:40.178607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66282d-dc29-464a-b63d-9db086d8494d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:40.179175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66282d-dc29-464a-b63d-9db086d8494d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:40.179175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66282d-dc29-464a-b63d-9db086d8494d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:40.200494 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-1c66282d-dc29-464a-b63d-9db086d8494d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:49:40.202338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1c66282d-dc29-464a-b63d-9db086d8494d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:40.210862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:49:40.497429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7df50-6d8d-410d-947a-97f5fcd2ebc8 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:49:40.499896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4fe7df50-6d8d-410d-947a-97f5fcd2ebc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:40.537066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:49:40.590313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7df50-6d8d-410d-947a-97f5fcd2ebc8 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:49:40.594510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa20f6-32e5-4c4d-901d-98a47078fc84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:49:40.595119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa20f6-32e5-4c4d-901d-98a47078fc84 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:40.595393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa20f6-32e5-4c4d-901d-98a47078fc84 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:40.595662 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa20f6-32e5-4c4d-901d-98a47078fc84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:40.605691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa20f6-32e5-4c4d-901d-98a47078fc84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:49:40.607900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa20f6-32e5-4c4d-901d-98a47078fc84 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:49:40.618171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 765/1530] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:49:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:49:42.625696 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759bdfb-ef29-4bab-a831-7d53561168ed None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:42.626232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759bdfb-ef29-4bab-a831-7d53561168ed None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:42.626232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759bdfb-ef29-4bab-a831-7d53561168ed None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:42.627846 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:49:42.633857 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca4bd2-f370-4efa-855e-75b798cc4af5 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:42.634360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca4bd2-f370-4efa-855e-75b798cc4af5 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:42.634478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca4bd2-f370-4efa-855e-75b798cc4af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:42.657253 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-00ca4bd2-f370-4efa-855e-75b798cc4af5 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:49:42.658473 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-00ca4bd2-f370-4efa-855e-75b798cc4af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:42.666536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:49:42.953080 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759bdfb-ef29-4bab-a831-7d53561168ed None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:49:42.964451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759bdfb-ef29-4bab-a831-7d53561168ed None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:49:42.968390 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704fec6-09fb-481f-956c-71847fbebc20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:49:42.968993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704fec6-09fb-481f-956c-71847fbebc20 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:42.969292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704fec6-09fb-481f-956c-71847fbebc20 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:42.969653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704fec6-09fb-481f-956c-71847fbebc20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:42.979700 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704fec6-09fb-481f-956c-71847fbebc20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:49:42.981169 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704fec6-09fb-481f-956c-71847fbebc20 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:49:42.991226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 767/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:49:46.540259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bce74-f774-48c5-a2e4-db9565ca4cfd None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:46.540889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bce74-f774-48c5-a2e4-db9565ca4cfd None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:46.540889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bce74-f774-48c5-a2e4-db9565ca4cfd None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:46.542138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:49:46.548236 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722fccf-284d-474a-92a7-c4e3003121f6 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:46.548794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722fccf-284d-474a-92a7-c4e3003121f6 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:46.548794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722fccf-284d-474a-92a7-c4e3003121f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:46.568382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-2722fccf-284d-474a-92a7-c4e3003121f6 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:49:46.569623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2722fccf-284d-474a-92a7-c4e3003121f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:46.578137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:49:47.143296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bce74-f774-48c5-a2e4-db9565ca4cfd None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:49:47.145060 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-283bce74-f774-48c5-a2e4-db9565ca4cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:47.226083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bce74-f774-48c5-a2e4-db9565ca4cfd None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:49:47.230362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a28d5e-11d5-493d-b685-880e23eeb8b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:49:47.230962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a28d5e-11d5-493d-b685-880e23eeb8b4 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:47.231236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a28d5e-11d5-493d-b685-880e23eeb8b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:47.231504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a28d5e-11d5-493d-b685-880e23eeb8b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:47.242285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a28d5e-11d5-493d-b685-880e23eeb8b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:49:47.243843 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a28d5e-11d5-493d-b685-880e23eeb8b4 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:49:47.256198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 768/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:49:49.229161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19b528-d93b-4a46-8cdf-2815a9efe74c None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:49.229686 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19b528-d93b-4a46-8cdf-2815a9efe74c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:49.229686 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19b528-d93b-4a46-8cdf-2815a9efe74c None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:49.230928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 14:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:49:49.237172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8afe4-06a9-47a4-b4d8-03b74b6239e6 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:49.237675 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8afe4-06a9-47a4-b4d8-03b74b6239e6 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:49.237778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8afe4-06a9-47a4-b4d8-03b74b6239e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:49.258017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8bc8afe4-06a9-47a4-b4d8-03b74b6239e6 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:49:49.259233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8bc8afe4-06a9-47a4-b4d8-03b74b6239e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:49.267007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 14:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:49:49.813425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19b528-d93b-4a46-8cdf-2815a9efe74c None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:49:49.815140 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-be19b528-d93b-4a46-8cdf-2815a9efe74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:49:49.828163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19b528-d93b-4a46-8cdf-2815a9efe74c None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:49:49.831666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7463377-76dc-4d46-bc7a-23e854930a89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:49:49.832302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7463377-76dc-4d46-bc7a-23e854930a89 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:49:49.832579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7463377-76dc-4d46-bc7a-23e854930a89 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:49:49.832846 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7463377-76dc-4d46-bc7a-23e854930a89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:49:49.842659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7463377-76dc-4d46-bc7a-23e854930a89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:49:49.844129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7463377-76dc-4d46-bc7a-23e854930a89 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:49:49.853871 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 770/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:49:59.252767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=63 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:49:59.849462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=37 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:15.111911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b83817-ca60-4b8f-aaf0-870c85830ae9 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.112515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b83817-ca60-4b8f-aaf0-870c85830ae9 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.112515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b83817-ca60-4b8f-aaf0-870c85830ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.135997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-a5b83817-ca60-4b8f-aaf0-870c85830ae9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.137670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a5b83817-ca60-4b8f-aaf0-870c85830ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.148743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 770/1540] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.159057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95943b38-6ca9-4056-8c0c-b3bf6155c210 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.159648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95943b38-6ca9-4056-8c0c-b3bf6155c210 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.159648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95943b38-6ca9-4056-8c0c-b3bf6155c210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.185203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29097a4-cee3-4c11-8b8e-20269380a08f None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.185879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29097a4-cee3-4c11-8b8e-20269380a08f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.186442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29097a4-cee3-4c11-8b8e-20269380a08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.192723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-95943b38-6ca9-4056-8c0c-b3bf6155c210 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.194248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-95943b38-6ca9-4056-8c0c-b3bf6155c210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.204960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 771/1541] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.206480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ac063-dcd4-49af-b3b2-9a74be75c68d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.212736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ac063-dcd4-49af-b3b2-9a74be75c68d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.213433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ac063-dcd4-49af-b3b2-9a74be75c68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.221348 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-e29097a4-cee3-4c11-8b8e-20269380a08f None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.222900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e29097a4-cee3-4c11-8b8e-20269380a08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.244750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 771/1542] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.246615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29097a4-cee3-4c11-8b8e-20269380a08f None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.259529 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a78ac063-dcd4-49af-b3b2-9a74be75c68d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.259529 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a78ac063-dcd4-49af-b3b2-9a74be75c68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.272113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 772/1543] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.273151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29097a4-cee3-4c11-8b8e-20269380a08f None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.273378 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881dcc76-dfcb-4ff6-914f-9eb502747b76 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.273670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881dcc76-dfcb-4ff6-914f-9eb502747b76 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.273948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881dcc76-dfcb-4ff6-914f-9eb502747b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.291400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54012447-8b0f-4313-9c66-67dffd7f3203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.292976 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012447-8b0f-4313-9c66-67dffd7f3203 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.293491 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012447-8b0f-4313-9c66-67dffd7f3203 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.294399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012447-8b0f-4313-9c66-67dffd7f3203 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.295744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012447-8b0f-4313-9c66-67dffd7f3203 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:15.296557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012447-8b0f-4313-9c66-67dffd7f3203 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.299587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012447-8b0f-4313-9c66-67dffd7f3203 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:15.301588 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-881dcc76-dfcb-4ff6-914f-9eb502747b76 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.308192 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-881dcc76-dfcb-4ff6-914f-9eb502747b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.318860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 773/1544] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.324325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95021593-0d44-45c2-baa9-e4b181413918 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.324692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95021593-0d44-45c2-baa9-e4b181413918 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.324980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95021593-0d44-45c2-baa9-e4b181413918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.329955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 772/1545] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:15.331815 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ce66d-14b2-475d-9d43-2629922977cd None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.332105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ce66d-14b2-475d-9d43-2629922977cd None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.332383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ce66d-14b2-475d-9d43-2629922977cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.367506 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-95021593-0d44-45c2-baa9-e4b181413918 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.368862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-95021593-0d44-45c2-baa9-e4b181413918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.373033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-2c4ce66d-14b2-475d-9d43-2629922977cd None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.374232 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2c4ce66d-14b2-475d-9d43-2629922977cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.378667 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 774/1546] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.379904 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27411b-0342-4448-a62e-bdaf24bd1430 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.380179 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27411b-0342-4448-a62e-bdaf24bd1430 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.380469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27411b-0342-4448-a62e-bdaf24bd1430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.384140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 773/1547] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.385465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a3430-9578-4b53-a3c9-ecf24ce2b8a9 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.385846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a3430-9578-4b53-a3c9-ecf24ce2b8a9 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.386181 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a3430-9578-4b53-a3c9-ecf24ce2b8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.404637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-3e27411b-0342-4448-a62e-bdaf24bd1430 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.405957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3e27411b-0342-4448-a62e-bdaf24bd1430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.407489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-1f6a3430-9578-4b53-a3c9-ecf24ce2b8a9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.408748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-1f6a3430-9578-4b53-a3c9-ecf24ce2b8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.415339 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 775/1548] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.416449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27411b-0342-4448-a62e-bdaf24bd1430 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.417569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 774/1549] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.418926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a403b-9153-41b9-a0cf-143210cce887 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.419271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a403b-9153-41b9-a0cf-143210cce887 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.419665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a403b-9153-41b9-a0cf-143210cce887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.430850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27411b-0342-4448-a62e-bdaf24bd1430 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.437297 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402cefd-4c96-4780-94af-597d16458a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.437906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402cefd-4c96-4780-94af-597d16458a43 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.438180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402cefd-4c96-4780-94af-597d16458a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.438449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402cefd-4c96-4780-94af-597d16458a43 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.439074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402cefd-4c96-4780-94af-597d16458a43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.441024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402cefd-4c96-4780-94af-597d16458a43 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:15.442753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-524a403b-9153-41b9-a0cf-143210cce887 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.443988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-524a403b-9153-41b9-a0cf-143210cce887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.452099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 775/1550] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.453323 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b193d-a3d6-4fed-8d8b-dbf535f5036c None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.453604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b193d-a3d6-4fed-8d8b-dbf535f5036c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.453875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b193d-a3d6-4fed-8d8b-dbf535f5036c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.476301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-ef3b193d-a3d6-4fed-8d8b-dbf535f5036c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.477525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ef3b193d-a3d6-4fed-8d8b-dbf535f5036c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.483711 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 776/1551] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:15.484506 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402cefd-4c96-4780-94af-597d16458a43 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.486991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 776/1552] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.487989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985aad7-0934-4965-8904-ab0a79e8d888 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.488309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985aad7-0934-4965-8904-ab0a79e8d888 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.488549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985aad7-0934-4965-8904-ab0a79e8d888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.509683 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8985aad7-0934-4965-8904-ab0a79e8d888 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.510913 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8985aad7-0934-4965-8904-ab0a79e8d888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.555025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402cefd-4c96-4780-94af-597d16458a43 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.559918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224f6a1-d326-448c-b869-92109e6ffce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.560528 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224f6a1-d326-448c-b869-92109e6ffce8 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.560814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224f6a1-d326-448c-b869-92109e6ffce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.561086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224f6a1-d326-448c-b869-92109e6ffce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.561728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224f6a1-d326-448c-b869-92109e6ffce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:15.562030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224f6a1-d326-448c-b869-92109e6ffce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.563046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: WARNING py.warnings [None req-d224f6a1-d326-448c-b869-92109e6ffce8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:51:15.563046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: warnings.warn(msg) Aug 18 14:51:15.563046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: Aug 18 14:51:15.563966 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224f6a1-d326-448c-b869-92109e6ffce8 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:15.570102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 777/1553] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.571031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985aad7-0934-4965-8904-ab0a79e8d888 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.577789 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 777/1554] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:15.577789 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224f6a1-d326-448c-b869-92109e6ffce8 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.582376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985aad7-0934-4965-8904-ab0a79e8d888 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.587132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f52dc-9f62-4e53-9476-b3cda88a62ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.587779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f52dc-9f62-4e53-9476-b3cda88a62ba admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.588086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f52dc-9f62-4e53-9476-b3cda88a62ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.588353 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f52dc-9f62-4e53-9476-b3cda88a62ba admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.588609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224f6a1-d326-448c-b869-92109e6ffce8 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.589618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f52dc-9f62-4e53-9476-b3cda88a62ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:15.589915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f52dc-9f62-4e53-9476-b3cda88a62ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.591508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f52dc-9f62-4e53-9476-b3cda88a62ba admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:15.594315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fd416-4c1d-4d3f-8eb7-6011f565ad4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.594554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fd416-4c1d-4d3f-8eb7-6011f565ad4d admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.594843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fd416-4c1d-4d3f-8eb7-6011f565ad4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.595113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fd416-4c1d-4d3f-8eb7-6011f565ad4d admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.595796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fd416-4c1d-4d3f-8eb7-6011f565ad4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:15.596104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fd416-4c1d-4d3f-8eb7-6011f565ad4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.597732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fd416-4c1d-4d3f-8eb7-6011f565ad4d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:15.610219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 778/1555] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:15.610219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63efe4d-21b9-45b4-bb0e-c70b14176b49 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.610219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63efe4d-21b9-45b4-bb0e-c70b14176b49 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.610219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63efe4d-21b9-45b4-bb0e-c70b14176b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.616376 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 778/1556] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:15.617807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a381a-d06a-44df-9573-68275b1de6de None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.618107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a381a-d06a-44df-9573-68275b1de6de None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.618471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a381a-d06a-44df-9573-68275b1de6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.633874 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-a63efe4d-21b9-45b4-bb0e-c70b14176b49 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.635107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a63efe4d-21b9-45b4-bb0e-c70b14176b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.643294 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 779/1557] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.644137 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63efe4d-21b9-45b4-bb0e-c70b14176b49 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.647663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-433a381a-d06a-44df-9573-68275b1de6de None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.649008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-433a381a-d06a-44df-9573-68275b1de6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.660262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 779/1558] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.661527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb81dab-6b59-49df-b204-8c91402f138d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.661816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb81dab-6b59-49df-b204-8c91402f138d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.662092 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb81dab-6b59-49df-b204-8c91402f138d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.662919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63efe4d-21b9-45b4-bb0e-c70b14176b49 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.668297 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd88649-5db4-4b0d-b0de-665f233d8037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.668894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd88649-5db4-4b0d-b0de-665f233d8037 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.669168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd88649-5db4-4b0d-b0de-665f233d8037 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.669438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd88649-5db4-4b0d-b0de-665f233d8037 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.670064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd88649-5db4-4b0d-b0de-665f233d8037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.672342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd88649-5db4-4b0d-b0de-665f233d8037 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:15.682039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-ccb81dab-6b59-49df-b204-8c91402f138d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.683321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ccb81dab-6b59-49df-b204-8c91402f138d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.691787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 780/1559] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.693025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb81dab-6b59-49df-b204-8c91402f138d None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.697529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 780/1560] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/users => generated 289 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:15.697529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd88649-5db4-4b0d-b0de-665f233d8037 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.703354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb81dab-6b59-49df-b204-8c91402f138d None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.705481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd88649-5db4-4b0d-b0de-665f233d8037 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.708133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1fd46e-2746-47d6-973d-f2a542f994fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.708738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fd46e-2746-47d6-973d-f2a542f994fa admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.709017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fd46e-2746-47d6-973d-f2a542f994fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.709287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fd46e-2746-47d6-973d-f2a542f994fa admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.709928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1fd46e-2746-47d6-973d-f2a542f994fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:15.710050 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4c126-107a-4130-848c-00d4d9aa8f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.710231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1fd46e-2746-47d6-973d-f2a542f994fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.710545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4c126-107a-4130-848c-00d4d9aa8f02 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.710821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4c126-107a-4130-848c-00d4d9aa8f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.711089 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4c126-107a-4130-848c-00d4d9aa8f02 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.711731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4c126-107a-4130-848c-00d4d9aa8f02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.711857 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1fd46e-2746-47d6-973d-f2a542f994fa admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:15.712747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: WARNING py.warnings [None req-21a4c126-107a-4130-848c-00d4d9aa8f02 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:51:15.712747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: warnings.warn(msg) Aug 18 14:51:15.712747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: Aug 18 14:51:15.714570 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4c126-107a-4130-848c-00d4d9aa8f02 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:15.732515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 781/1561] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:15.732515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1fd46e-2746-47d6-973d-f2a542f994fa admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.736505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1fd46e-2746-47d6-973d-f2a542f994fa admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.743135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa31e8-0bae-48f7-8038-c80156da8907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.744038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa31e8-0bae-48f7-8038-c80156da8907 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.744449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa31e8-0bae-48f7-8038-c80156da8907 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.744737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa31e8-0bae-48f7-8038-c80156da8907 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.745428 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa31e8-0bae-48f7-8038-c80156da8907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.747365 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa31e8-0bae-48f7-8038-c80156da8907 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:15.752665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 781/1562] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:15.753447 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4c126-107a-4130-848c-00d4d9aa8f02 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.781272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 782/1563] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:15.781272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a0349-ea7d-4556-a7c6-d48a621d575b None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.781272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a0349-ea7d-4556-a7c6-d48a621d575b None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.781272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a0349-ea7d-4556-a7c6-d48a621d575b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.799251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-2b9a0349-ea7d-4556-a7c6-d48a621d575b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:15.800302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2b9a0349-ea7d-4556-a7c6-d48a621d575b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:15.805802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:15.821018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4c126-107a-4130-848c-00d4d9aa8f02 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.825889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:15.862986 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 783/1564] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:15.863885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a0349-ea7d-4556-a7c6-d48a621d575b None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.873544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a0349-ea7d-4556-a7c6-d48a621d575b None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.878296 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803b1e9-046b-497d-a009-edc9759dd70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.878900 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803b1e9-046b-497d-a009-edc9759dd70a admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.879182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803b1e9-046b-497d-a009-edc9759dd70a admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.879462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803b1e9-046b-497d-a009-edc9759dd70a admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.880145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803b1e9-046b-497d-a009-edc9759dd70a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:15.880450 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803b1e9-046b-497d-a009-edc9759dd70a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.880789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03678582-0095-4f71-92dc-c59adbc6ffaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.881742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03678582-0095-4f71-92dc-c59adbc6ffaf admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.881938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03678582-0095-4f71-92dc-c59adbc6ffaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.882205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03678582-0095-4f71-92dc-c59adbc6ffaf admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.882461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803b1e9-046b-497d-a009-edc9759dd70a admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:15.882826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03678582-0095-4f71-92dc-c59adbc6ffaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.884747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03678582-0095-4f71-92dc-c59adbc6ffaf admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:15.895632 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 784/1565] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:15.895896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803b1e9-046b-497d-a009-edc9759dd70a admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.911354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803b1e9-046b-497d-a009-edc9759dd70a admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.918321 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 782/1566] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:15.918321 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03678582-0095-4f71-92dc-c59adbc6ffaf admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.919512 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a353d-4db3-4440-aeb3-ce4ce14f327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.920267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a353d-4db3-4440-aeb3-ce4ce14f327e admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.920546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a353d-4db3-4440-aeb3-ce4ce14f327e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.920845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a353d-4db3-4440-aeb3-ce4ce14f327e admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.921473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a353d-4db3-4440-aeb3-ce4ce14f327e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:15.921771 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a353d-4db3-4440-aeb3-ce4ce14f327e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.924337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a353d-4db3-4440-aeb3-ce4ce14f327e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:15.938065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 785/1567] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:15.938692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a353d-4db3-4440-aeb3-ce4ce14f327e admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:15.948162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a353d-4db3-4440-aeb3-ce4ce14f327e admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.985789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03678582-0095-4f71-92dc-c59adbc6ffaf admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:15.990577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ebe79-81fe-4021-ad7a-f9c69bc7b5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:15.991189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ebe79-81fe-4021-ad7a-f9c69bc7b5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:15.991462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ebe79-81fe-4021-ad7a-f9c69bc7b5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:15.991763 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ebe79-81fe-4021-ad7a-f9c69bc7b5c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:15.992399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ebe79-81fe-4021-ad7a-f9c69bc7b5c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:15.993994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ebe79-81fe-4021-ad7a-f9c69bc7b5c6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.007286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 783/1568] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.007286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ebe79-81fe-4021-ad7a-f9c69bc7b5c6 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.007600 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bebd3-a5f7-4108-89fb-e1ab893fb27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.007600 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bebd3-a5f7-4108-89fb-e1ab893fb27f admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.007600 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bebd3-a5f7-4108-89fb-e1ab893fb27f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.007600 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bebd3-a5f7-4108-89fb-e1ab893fb27f admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.008005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bebd3-a5f7-4108-89fb-e1ab893fb27f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.009596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bebd3-a5f7-4108-89fb-e1ab893fb27f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.015984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ebe79-81fe-4021-ad7a-f9c69bc7b5c6 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.020491 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da2b67-5071-4eb0-a38b-46ed68356e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.021085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da2b67-5071-4eb0-a38b-46ed68356e8c admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.021352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da2b67-5071-4eb0-a38b-46ed68356e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.021624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da2b67-5071-4eb0-a38b-46ed68356e8c admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.022235 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da2b67-5071-4eb0-a38b-46ed68356e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.024168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da2b67-5071-4eb0-a38b-46ed68356e8c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.041371 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 786/1569] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:15 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:16.041371 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bebd3-a5f7-4108-89fb-e1ab893fb27f admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.048180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 784/1570] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:16 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:16.052332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da2b67-5071-4eb0-a38b-46ed68356e8c admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.062222 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da2b67-5071-4eb0-a38b-46ed68356e8c admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.101078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bebd3-a5f7-4108-89fb-e1ab893fb27f admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.105939 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d559c7b-795b-4ed1-a582-09b4de795e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.106545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d559c7b-795b-4ed1-a582-09b4de795e06 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.106820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d559c7b-795b-4ed1-a582-09b4de795e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.107087 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d559c7b-795b-4ed1-a582-09b4de795e06 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.107730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d559c7b-795b-4ed1-a582-09b4de795e06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.108739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: WARNING py.warnings [None req-7d559c7b-795b-4ed1-a582-09b4de795e06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:51:16.108739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: warnings.warn(msg) Aug 18 14:51:16.108739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: Aug 18 14:51:16.109942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d559c7b-795b-4ed1-a582-09b4de795e06 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.121778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d2c0b-80d2-405c-92d6-387a6d760dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.122849 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d2c0b-80d2-405c-92d6-387a6d760dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.123362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d2c0b-80d2-405c-92d6-387a6d760dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.123981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d2c0b-80d2-405c-92d6-387a6d760dc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.124761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d2c0b-80d2-405c-92d6-387a6d760dc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:16.125163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d2c0b-80d2-405c-92d6-387a6d760dc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.127166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d2c0b-80d2-405c-92d6-387a6d760dc2 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.129251 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:16.139668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 787/1571] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:16 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:16.140820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d559c7b-795b-4ed1-a582-09b4de795e06 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.143473 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 785/1572] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:16.144236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d2c0b-80d2-405c-92d6-387a6d760dc2 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.153948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d559c7b-795b-4ed1-a582-09b4de795e06 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.155977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d2c0b-80d2-405c-92d6-387a6d760dc2 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.159011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f036695-2c4b-4b5f-a4ab-7ed0131a2202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.159626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f036695-2c4b-4b5f-a4ab-7ed0131a2202 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.160238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f036695-2c4b-4b5f-a4ab-7ed0131a2202 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.160347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f036695-2c4b-4b5f-a4ab-7ed0131a2202 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.160988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f036695-2c4b-4b5f-a4ab-7ed0131a2202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.161106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94464a-f5f5-401a-80e7-1199aa80cf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.161684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94464a-f5f5-401a-80e7-1199aa80cf06 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.161960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94464a-f5f5-401a-80e7-1199aa80cf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.162259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94464a-f5f5-401a-80e7-1199aa80cf06 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.162896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94464a-f5f5-401a-80e7-1199aa80cf06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.163005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f036695-2c4b-4b5f-a4ab-7ed0131a2202 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.164826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94464a-f5f5-401a-80e7-1199aa80cf06 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.179825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:16.188809 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 786/1573] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:16 2022] POST /identity/v3/users => generated 286 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:16.190017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94464a-f5f5-401a-80e7-1199aa80cf06 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.195490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 788/1574] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:16 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:16.196352 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f036695-2c4b-4b5f-a4ab-7ed0131a2202 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.205143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94464a-f5f5-401a-80e7-1199aa80cf06 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.259442 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f036695-2c4b-4b5f-a4ab-7ed0131a2202 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.264796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a1fba-a57f-48ed-b0a3-ad181428a327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.265407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a1fba-a57f-48ed-b0a3-ad181428a327 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.265684 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a1fba-a57f-48ed-b0a3-ad181428a327 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.265961 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a1fba-a57f-48ed-b0a3-ad181428a327 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.266599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a1fba-a57f-48ed-b0a3-ad181428a327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.268241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a1fba-a57f-48ed-b0a3-ad181428a327 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.269890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe86eeb-0928-4ca6-bb7a-07c8bd8393ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.270775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe86eeb-0928-4ca6-bb7a-07c8bd8393ac admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.271166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe86eeb-0928-4ca6-bb7a-07c8bd8393ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.271544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe86eeb-0928-4ca6-bb7a-07c8bd8393ac admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.272543 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe86eeb-0928-4ca6-bb7a-07c8bd8393ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.275519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe86eeb-0928-4ca6-bb7a-07c8bd8393ac admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.278156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 789/1575] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.278925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a1fba-a57f-48ed-b0a3-ad181428a327 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.288338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a1fba-a57f-48ed-b0a3-ad181428a327 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.293652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db1fd5-d07c-4923-8a9a-875a760accc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.294253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db1fd5-d07c-4923-8a9a-875a760accc7 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.294529 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db1fd5-d07c-4923-8a9a-875a760accc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.294798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db1fd5-d07c-4923-8a9a-875a760accc7 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.295491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db1fd5-d07c-4923-8a9a-875a760accc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.297627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db1fd5-d07c-4923-8a9a-875a760accc7 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.304940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 787/1576] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:16 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:16.306075 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe86eeb-0928-4ca6-bb7a-07c8bd8393ac admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.320909 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 790/1577] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:16 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:16.321748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db1fd5-d07c-4923-8a9a-875a760accc7 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.332224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db1fd5-d07c-4923-8a9a-875a760accc7 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.378862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe86eeb-0928-4ca6-bb7a-07c8bd8393ac admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.383867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9610805-d940-4073-b174-85c1f8a13b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.384474 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9610805-d940-4073-b174-85c1f8a13b08 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.384754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9610805-d940-4073-b174-85c1f8a13b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.385023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9610805-d940-4073-b174-85c1f8a13b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.386364 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9610805-d940-4073-b174-85c1f8a13b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.387988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9610805-d940-4073-b174-85c1f8a13b08 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.392378 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f233363-caef-4044-b791-6063b314c082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.392995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f233363-caef-4044-b791-6063b314c082 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.393273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f233363-caef-4044-b791-6063b314c082 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.393543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f233363-caef-4044-b791-6063b314c082 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.394187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f233363-caef-4044-b791-6063b314c082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.395818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f233363-caef-4044-b791-6063b314c082 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.398393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 788/1578] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.399272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9610805-d940-4073-b174-85c1f8a13b08 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.409519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9610805-d940-4073-b174-85c1f8a13b08 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.409767 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 791/1579] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.410633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f233363-caef-4044-b791-6063b314c082 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.417294 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7352256-2444-4aa4-8639-c125cb46e68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.417939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7352256-2444-4aa4-8639-c125cb46e68a admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.418218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7352256-2444-4aa4-8639-c125cb46e68a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.418514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7352256-2444-4aa4-8639-c125cb46e68a admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.419159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7352256-2444-4aa4-8639-c125cb46e68a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.420688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f233363-caef-4044-b791-6063b314c082 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.421246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7352256-2444-4aa4-8639-c125cb46e68a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.425843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502af36-b0e8-4766-8203-394ee744227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.426455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502af36-b0e8-4766-8203-394ee744227c admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.426732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502af36-b0e8-4766-8203-394ee744227c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.427143 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502af36-b0e8-4766-8203-394ee744227c admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.427818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502af36-b0e8-4766-8203-394ee744227c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.429396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502af36-b0e8-4766-8203-394ee744227c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.440614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 792/1580] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.441453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502af36-b0e8-4766-8203-394ee744227c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.452595 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502af36-b0e8-4766-8203-394ee744227c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.458372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 789/1581] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:16 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:16.458372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7352256-2444-4aa4-8639-c125cb46e68a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.460796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017043c-6fb5-44ae-8300-87222b1608a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.461432 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017043c-6fb5-44ae-8300-87222b1608a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.461716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017043c-6fb5-44ae-8300-87222b1608a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.461992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017043c-6fb5-44ae-8300-87222b1608a1 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.462614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017043c-6fb5-44ae-8300-87222b1608a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.464691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017043c-6fb5-44ae-8300-87222b1608a1 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.487992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 793/1582] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:16 2022] POST /identity/v3/users => generated 283 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:16.488810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017043c-6fb5-44ae-8300-87222b1608a1 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.500893 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017043c-6fb5-44ae-8300-87222b1608a1 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.501590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:16.527945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7352256-2444-4aa4-8639-c125cb46e68a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.532644 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc9ad8-eca6-4f71-b494-99233fcd5ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.533419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc9ad8-eca6-4f71-b494-99233fcd5ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.533716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc9ad8-eca6-4f71-b494-99233fcd5ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.533993 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc9ad8-eca6-4f71-b494-99233fcd5ab3 admin admin] PATH_INFO: `/v3/projects/bde4e338430442449fcc203cd9493bc3/users/4d96208474754b2381d769d807689bbb/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.546527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:16.550074 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc9ad8-eca6-4f71-b494-99233fcd5ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde4e338430442449fcc203cd9493bc3, user_id=4d96208474754b2381d769d807689bbb, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.551142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: WARNING py.warnings [None req-b4bc9ad8-eca6-4f71-b494-99233fcd5ab3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:51:16.551142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: warnings.warn(msg) Aug 18 14:51:16.551142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: Aug 18 14:51:16.553327 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc9ad8-eca6-4f71-b494-99233fcd5ab3 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.565250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b24143-f778-4de2-b4e4-0a7724912f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.566058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b24143-f778-4de2-b4e4-0a7724912f3a admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.566332 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b24143-f778-4de2-b4e4-0a7724912f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.566601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b24143-f778-4de2-b4e4-0a7724912f3a admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.567370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b24143-f778-4de2-b4e4-0a7724912f3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.567480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 790/1583] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:16 2022] PUT /identity/v3/projects/bde4e338430442449fcc203cd9493bc3/users/4d96208474754b2381d769d807689bbb/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:16.568415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc9ad8-eca6-4f71-b494-99233fcd5ab3 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.572970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b24143-f778-4de2-b4e4-0a7724912f3a admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.598244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 794/1584] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:16 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:16.598244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b24143-f778-4de2-b4e4-0a7724912f3a admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.606701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b24143-f778-4de2-b4e4-0a7724912f3a admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.645728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc9ad8-eca6-4f71-b494-99233fcd5ab3 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.650415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0210bcc-2516-4a7b-a497-f3204780698f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.650718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0210bcc-2516-4a7b-a497-f3204780698f admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.650996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0210bcc-2516-4a7b-a497-f3204780698f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.651295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0210bcc-2516-4a7b-a497-f3204780698f admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.651969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0210bcc-2516-4a7b-a497-f3204780698f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.653591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0210bcc-2516-4a7b-a497-f3204780698f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.665190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 791/1585] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.666188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0210bcc-2516-4a7b-a497-f3204780698f admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.677017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0210bcc-2516-4a7b-a497-f3204780698f admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.681675 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d5532-d440-4dc6-acd4-9207b8844b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.681961 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62564f8e-68ed-4993-bb7a-cc38554a48e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.682267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d5532-d440-4dc6-acd4-9207b8844b01 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.683139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62564f8e-68ed-4993-bb7a-cc38554a48e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.683282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d5532-d440-4dc6-acd4-9207b8844b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.683612 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d5532-d440-4dc6-acd4-9207b8844b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.684241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62564f8e-68ed-4993-bb7a-cc38554a48e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.684241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62564f8e-68ed-4993-bb7a-cc38554a48e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.684476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d5532-d440-4dc6-acd4-9207b8844b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.684701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62564f8e-68ed-4993-bb7a-cc38554a48e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.685821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d5532-d440-4dc6-acd4-9207b8844b01 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.686264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62564f8e-68ed-4993-bb7a-cc38554a48e8 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.697060 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 792/1587] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.697243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 795/1587] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.697699 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d5532-d440-4dc6-acd4-9207b8844b01 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.698525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62564f8e-68ed-4993-bb7a-cc38554a48e8 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.708776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62564f8e-68ed-4993-bb7a-cc38554a48e8 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.710245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d5532-d440-4dc6-acd4-9207b8844b01 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.713407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2d206-f971-4dc6-a6a8-00612fd8755b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.714088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2d206-f971-4dc6-a6a8-00612fd8755b admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.714358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2d206-f971-4dc6-a6a8-00612fd8755b admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.714662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2d206-f971-4dc6-a6a8-00612fd8755b admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.715191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2d206-f971-4dc6-a6a8-00612fd8755b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.715495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926a7e4-d6d9-467c-847d-1dfa4e675ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.716144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926a7e4-d6d9-467c-847d-1dfa4e675ddb admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.716415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926a7e4-d6d9-467c-847d-1dfa4e675ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.716731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926a7e4-d6d9-467c-847d-1dfa4e675ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.717152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2d206-f971-4dc6-a6a8-00612fd8755b admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.717315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926a7e4-d6d9-467c-847d-1dfa4e675ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.718875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926a7e4-d6d9-467c-847d-1dfa4e675ddb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.731641 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 793/1588] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.732832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926a7e4-d6d9-467c-847d-1dfa4e675ddb admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.743784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 796/1589] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:16 2022] POST /identity/v3/users => generated 292 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:16.743784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd6a22-1e0f-4b16-8630-76096601f854 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.743784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd6a22-1e0f-4b16-8630-76096601f854 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.743784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd6a22-1e0f-4b16-8630-76096601f854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.748890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926a7e4-d6d9-467c-847d-1dfa4e675ddb admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.753837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78083590-eb77-486c-b93d-2215c6fab1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.754438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78083590-eb77-486c-b93d-2215c6fab1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.754717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78083590-eb77-486c-b93d-2215c6fab1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.754989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78083590-eb77-486c-b93d-2215c6fab1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.755631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78083590-eb77-486c-b93d-2215c6fab1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.757538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78083590-eb77-486c-b93d-2215c6fab1d5 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.765058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-5ddd6a22-1e0f-4b16-8630-76096601f854 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:16.766290 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5ddd6a22-1e0f-4b16-8630-76096601f854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:16.771345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 794/1590] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.771345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78083590-eb77-486c-b93d-2215c6fab1d5 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.776793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 797/1591] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:16.777628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd6a22-1e0f-4b16-8630-76096601f854 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.783640 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78083590-eb77-486c-b93d-2215c6fab1d5 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.787366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd6a22-1e0f-4b16-8630-76096601f854 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.788440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10c349-4924-4dbd-a907-f51299947fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.789108 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10c349-4924-4dbd-a907-f51299947fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.789412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10c349-4924-4dbd-a907-f51299947fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.789679 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10c349-4924-4dbd-a907-f51299947fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.790446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10c349-4924-4dbd-a907-f51299947fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.792634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10c349-4924-4dbd-a907-f51299947fa9 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.792897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f6c5e-1894-473c-b186-97eba3d2e0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.793510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f6c5e-1894-473c-b186-97eba3d2e0a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.793782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f6c5e-1894-473c-b186-97eba3d2e0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.794086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f6c5e-1894-473c-b186-97eba3d2e0a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.794734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f6c5e-1894-473c-b186-97eba3d2e0a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.796367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f6c5e-1894-473c-b186-97eba3d2e0a6 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.804848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 795/1592] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.805501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10c349-4924-4dbd-a907-f51299947fa9 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.807175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 798/1593] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.809368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f6c5e-1894-473c-b186-97eba3d2e0a6 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.819964 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f6c5e-1894-473c-b186-97eba3d2e0a6 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.820177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10c349-4924-4dbd-a907-f51299947fa9 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.824983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce7fa3-c91b-4335-b3b8-ab5f8f4ffc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.825191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8eaf33-0456-40b4-85af-6873fdbe3ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.825642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce7fa3-c91b-4335-b3b8-ab5f8f4ffc8f admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.825917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8eaf33-0456-40b4-85af-6873fdbe3ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.826023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce7fa3-c91b-4335-b3b8-ab5f8f4ffc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.826209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce7fa3-c91b-4335-b3b8-ab5f8f4ffc8f admin admin] PATH_INFO: `/v3/projects/55c9fe0cbbfa47bca81668c039afc092/users/36dbc938bcd741f2a2b2013b6aa6f008/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.826336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8eaf33-0456-40b4-85af-6873fdbe3ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.826513 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8eaf33-0456-40b4-85af-6873fdbe3ccc admin admin] PATH_INFO: `/v3/projects/2a38dab41aca4c2d9258599840ce93d5/users/1469a960bd9241008cfb112a28bcd5d3/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.843229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce7fa3-c91b-4335-b3b8-ab5f8f4ffc8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c9fe0cbbfa47bca81668c039afc092, user_id=36dbc938bcd741f2a2b2013b6aa6f008, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.843621 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8eaf33-0456-40b4-85af-6873fdbe3ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a38dab41aca4c2d9258599840ce93d5, user_id=1469a960bd9241008cfb112a28bcd5d3, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.845864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce7fa3-c91b-4335-b3b8-ab5f8f4ffc8f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.846085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8eaf33-0456-40b4-85af-6873fdbe3ccc admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.861040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 799/1594] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:16 2022] PUT /identity/v3/projects/2a38dab41aca4c2d9258599840ce93d5/users/1469a960bd9241008cfb112a28bcd5d3/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:16.861850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8eaf33-0456-40b4-85af-6873fdbe3ccc admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.862444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 796/1595] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:16 2022] PUT /identity/v3/projects/55c9fe0cbbfa47bca81668c039afc092/users/36dbc938bcd741f2a2b2013b6aa6f008/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:16.863453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce7fa3-c91b-4335-b3b8-ab5f8f4ffc8f admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.934897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8eaf33-0456-40b4-85af-6873fdbe3ccc admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.938200 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:16.948781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:16.953849 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce7fa3-c91b-4335-b3b8-ab5f8f4ffc8f admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.959476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761f3d2-08e7-4217-857a-44cef7fed166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.960302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761f3d2-08e7-4217-857a-44cef7fed166 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.960639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761f3d2-08e7-4217-857a-44cef7fed166 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.961125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761f3d2-08e7-4217-857a-44cef7fed166 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.961957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761f3d2-08e7-4217-857a-44cef7fed166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.965187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761f3d2-08e7-4217-857a-44cef7fed166 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:16.978851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 797/1596] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:16.978851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761f3d2-08e7-4217-857a-44cef7fed166 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:16.986996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761f3d2-08e7-4217-857a-44cef7fed166 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:16.993055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b60e1-e477-4c64-96ec-14d1cc647411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:16.993798 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b60e1-e477-4c64-96ec-14d1cc647411 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:16.994273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b60e1-e477-4c64-96ec-14d1cc647411 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:16.994554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b60e1-e477-4c64-96ec-14d1cc647411 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:16.996389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b60e1-e477-4c64-96ec-14d1cc647411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:16.999466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b60e1-e477-4c64-96ec-14d1cc647411 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:17.003901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726c27c-f9a2-4a59-a59a-f310f4d9d4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:17.005000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726c27c-f9a2-4a59-a59a-f310f4d9d4db admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.005493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726c27c-f9a2-4a59-a59a-f310f4d9d4db admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.006227 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726c27c-f9a2-4a59-a59a-f310f4d9d4db admin admin] PATH_INFO: `/v3/projects/cf6c45f9df6f4fdb809218830afbacbc/users/1bdc7af040cb4449b556125327a0d9aa/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.014278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 798/1597] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:17.015138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b60e1-e477-4c64-96ec-14d1cc647411 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:17.027727 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726c27c-f9a2-4a59-a59a-f310f4d9d4db admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6c45f9df6f4fdb809218830afbacbc, user_id=1bdc7af040cb4449b556125327a0d9aa, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:17.030812 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726c27c-f9a2-4a59-a59a-f310f4d9d4db admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:17.046321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 800/1598] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:16 2022] PUT /identity/v3/projects/cf6c45f9df6f4fdb809218830afbacbc/users/1bdc7af040cb4449b556125327a0d9aa/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:17.047530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726c27c-f9a2-4a59-a59a-f310f4d9d4db admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:17.093334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b60e1-e477-4c64-96ec-14d1cc647411 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:17.114322 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726c27c-f9a2-4a59-a59a-f310f4d9d4db admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:17.119521 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cabf94-c327-4c7c-b1ae-c212f18cc8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:17.120235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cabf94-c327-4c7c-b1ae-c212f18cc8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.120512 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cabf94-c327-4c7c-b1ae-c212f18cc8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.120782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cabf94-c327-4c7c-b1ae-c212f18cc8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.121423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cabf94-c327-4c7c-b1ae-c212f18cc8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:17.123061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cabf94-c327-4c7c-b1ae-c212f18cc8f6 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:17.133764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 801/1599] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:17.134578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cabf94-c327-4c7c-b1ae-c212f18cc8f6 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:17.149530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce821bd5-b1b5-41f0-97d1-a370eb96d371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:17.150174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce821bd5-b1b5-41f0-97d1-a370eb96d371 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.150420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cabf94-c327-4c7c-b1ae-c212f18cc8f6 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:17.150638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce821bd5-b1b5-41f0-97d1-a370eb96d371 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.151006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce821bd5-b1b5-41f0-97d1-a370eb96d371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.156003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305454e-84ee-45d6-ade4-30108e693899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:17.156659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305454e-84ee-45d6-ade4-30108e693899 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.156936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305454e-84ee-45d6-ade4-30108e693899 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.157190 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305454e-84ee-45d6-ade4-30108e693899 admin admin] PATH_INFO: `/v3/projects/594f04dadf2a45fcb5fe2a80744cb9e6/users/09b055bb15ce4b2ab4276b0090e16122/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.161928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce821bd5-b1b5-41f0-97d1-a370eb96d371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:17.163378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce821bd5-b1b5-41f0-97d1-a370eb96d371 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:17.174014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305454e-84ee-45d6-ade4-30108e693899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=594f04dadf2a45fcb5fe2a80744cb9e6, user_id=09b055bb15ce4b2ab4276b0090e16122, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:17.174552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 799/1600] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:17.175323 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ffb89-a7c0-4d98-89ae-5102bd90bc87 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.175606 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ffb89-a7c0-4d98-89ae-5102bd90bc87 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.175912 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ffb89-a7c0-4d98-89ae-5102bd90bc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.177071 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305454e-84ee-45d6-ade4-30108e693899 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:17.199659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 802/1601] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:17 2022] PUT /identity/v3/projects/594f04dadf2a45fcb5fe2a80744cb9e6/users/09b055bb15ce4b2ab4276b0090e16122/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:17.199659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b2afd-3e18-450a-8204-7114780c54dd None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.199659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b2afd-3e18-450a-8204-7114780c54dd None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.200055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8e0ffb89-a7c0-4d98-89ae-5102bd90bc87 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:17.200141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b2afd-3e18-450a-8204-7114780c54dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.200216 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8e0ffb89-a7c0-4d98-89ae-5102bd90bc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:17.223971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-ca3b2afd-3e18-450a-8204-7114780c54dd None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:17.225636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ca3b2afd-3e18-450a-8204-7114780c54dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:17.283712 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 800/1602] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:17.285244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ffb89-a7c0-4d98-89ae-5102bd90bc87 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:17.305977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ffb89-a7c0-4d98-89ae-5102bd90bc87 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:17.309746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 803/1603] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:17.310474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b2afd-3e18-450a-8204-7114780c54dd None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:17.314157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce7108-a2aa-4fc7-bb69-1fb275632bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:17.315863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce7108-a2aa-4fc7-bb69-1fb275632bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.316643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce7108-a2aa-4fc7-bb69-1fb275632bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.317151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce7108-a2aa-4fc7-bb69-1fb275632bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.318702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce7108-a2aa-4fc7-bb69-1fb275632bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:17.321461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce7108-a2aa-4fc7-bb69-1fb275632bf1 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:17.326450 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b2afd-3e18-450a-8204-7114780c54dd None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:17.335396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab7ac5-a59b-4dd9-a188-322ab6a9f11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:17.336685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab7ac5-a59b-4dd9-a188-322ab6a9f11b admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.337285 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab7ac5-a59b-4dd9-a188-322ab6a9f11b admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.342141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab7ac5-a59b-4dd9-a188-322ab6a9f11b admin admin] PATH_INFO: `/v3/projects/224e153875984ef291dda96266f56f81/users/e3792e07c7ec49b78b3e202858a5b84f/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.342272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 801/1604] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:17.342272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b422b-be9d-4af6-924a-a30d2396e1ac None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.342272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b422b-be9d-4af6-924a-a30d2396e1ac None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.342272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b422b-be9d-4af6-924a-a30d2396e1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.352611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:17.363402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab7ac5-a59b-4dd9-a188-322ab6a9f11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=224e153875984ef291dda96266f56f81, user_id=e3792e07c7ec49b78b3e202858a5b84f, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:17.367029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab7ac5-a59b-4dd9-a188-322ab6a9f11b admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:17.377341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-829b422b-be9d-4af6-924a-a30d2396e1ac None None] MFA Rules not processed for user `1bdc7af040cb4449b556125327a0d9aa`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:17.379368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-829b422b-be9d-4af6-924a-a30d2396e1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:17.387126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 804/1605] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:17 2022] PUT /identity/v3/projects/224e153875984ef291dda96266f56f81/users/e3792e07c7ec49b78b3e202858a5b84f/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:17.388030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab7ac5-a59b-4dd9-a188-322ab6a9f11b admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:17.394274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:17.468288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab7ac5-a59b-4dd9-a188-322ab6a9f11b admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:17.473710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8d1f5-2e44-4a84-87ff-62f7d756cbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:17.474495 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8d1f5-2e44-4a84-87ff-62f7d756cbc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.474866 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8d1f5-2e44-4a84-87ff-62f7d756cbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.475235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8d1f5-2e44-4a84-87ff-62f7d756cbc3 admin admin] PATH_INFO: `/v3/projects/7e523324eb6745e0a8b6f2c147c91312/users/5f0c269f8bfd426b9eb18e011168040b/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.491184 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8d1f5-2e44-4a84-87ff-62f7d756cbc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e523324eb6745e0a8b6f2c147c91312, user_id=5f0c269f8bfd426b9eb18e011168040b, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:17.495754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 802/1606] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:17.495926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8d1f5-2e44-4a84-87ff-62f7d756cbc3 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:17.496025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fa486-4a60-4e40-b05b-db9b8a12a0ad None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.496025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fa486-4a60-4e40-b05b-db9b8a12a0ad None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.496025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fa486-4a60-4e40-b05b-db9b8a12a0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.514675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 805/1607] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:17 2022] PUT /identity/v3/projects/7e523324eb6745e0a8b6f2c147c91312/users/5f0c269f8bfd426b9eb18e011168040b/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:17.515579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8d1f5-2e44-4a84-87ff-62f7d756cbc3 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:17.519502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-421fa486-4a60-4e40-b05b-db9b8a12a0ad None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:17.520914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-421fa486-4a60-4e40-b05b-db9b8a12a0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:17.595761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8d1f5-2e44-4a84-87ff-62f7d756cbc3 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:17.600910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212a748-dad1-4045-b006-0f3f594e4af7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:17.601665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212a748-dad1-4045-b006-0f3f594e4af7 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.602040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212a748-dad1-4045-b006-0f3f594e4af7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.607425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212a748-dad1-4045-b006-0f3f594e4af7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.607667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 803/1608] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:17.607667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fa486-4a60-4e40-b05b-db9b8a12a0ad None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:17.607667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-421fa486-4a60-4e40-b05b-db9b8a12a0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:17.618379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fa486-4a60-4e40-b05b-db9b8a12a0ad None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:17.625794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e191c9-cd12-4338-8f06-1f328f1c5536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:17.627365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e191c9-cd12-4338-8f06-1f328f1c5536 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.628040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e191c9-cd12-4338-8f06-1f328f1c5536 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.628249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212a748-dad1-4045-b006-0f3f594e4af7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:17.628898 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e191c9-cd12-4338-8f06-1f328f1c5536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.629787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212a748-dad1-4045-b006-0f3f594e4af7 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:17.643357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 806/1609] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:17.644306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212a748-dad1-4045-b006-0f3f594e4af7 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:17.647971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e191c9-cd12-4338-8f06-1f328f1c5536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:17.650193 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e191c9-cd12-4338-8f06-1f328f1c5536 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:17.655029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212a748-dad1-4045-b006-0f3f594e4af7 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:17.662164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803a017-46bb-47a7-a6dc-bb8761baf213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:17.662897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803a017-46bb-47a7-a6dc-bb8761baf213 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.663177 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803a017-46bb-47a7-a6dc-bb8761baf213 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.663455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803a017-46bb-47a7-a6dc-bb8761baf213 admin admin] PATH_INFO: `/v3/projects/aded651ca7894034b7495f2fc3d58f51/users/ad318bf4dd64445496bd87d0bd64f3e2/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.666422 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 804/1610] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:17.668710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b7c5e-e82e-419a-a4cc-5bcc45c72a17 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.669307 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b7c5e-e82e-419a-a4cc-5bcc45c72a17 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.669808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b7c5e-e82e-419a-a4cc-5bcc45c72a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.684839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803a017-46bb-47a7-a6dc-bb8761baf213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aded651ca7894034b7495f2fc3d58f51, user_id=ad318bf4dd64445496bd87d0bd64f3e2, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:17.688179 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803a017-46bb-47a7-a6dc-bb8761baf213 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:17.714357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-730b7c5e-e82e-419a-a4cc-5bcc45c72a17 None None] MFA Rules not processed for user `e3792e07c7ec49b78b3e202858a5b84f`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:17.717224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-730b7c5e-e82e-419a-a4cc-5bcc45c72a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:17.721998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 807/1611] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:17 2022] PUT /identity/v3/projects/aded651ca7894034b7495f2fc3d58f51/users/ad318bf4dd64445496bd87d0bd64f3e2/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:17.723064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803a017-46bb-47a7-a6dc-bb8761baf213 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:17.824340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:17.844647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803a017-46bb-47a7-a6dc-bb8761baf213 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:17.852613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05af409-fd83-4ae8-af9c-8bec23f207fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:17.853734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05af409-fd83-4ae8-af9c-8bec23f207fb admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.854187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05af409-fd83-4ae8-af9c-8bec23f207fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.854780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05af409-fd83-4ae8-af9c-8bec23f207fb admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.855633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05af409-fd83-4ae8-af9c-8bec23f207fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:17.856878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 805/1612] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:17.858278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05af409-fd83-4ae8-af9c-8bec23f207fb admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:17.859124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfa42d-1ff7-4124-942a-089562170c5d None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:17.859696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfa42d-1ff7-4124-942a-089562170c5d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:17.860140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfa42d-1ff7-4124-942a-089562170c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:17.899508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 808/1613] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:17 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:17.900115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05af409-fd83-4ae8-af9c-8bec23f207fb admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:17.906517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-76cfa42d-1ff7-4124-942a-089562170c5d None None] MFA Rules not processed for user `5f0c269f8bfd426b9eb18e011168040b`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:17.908122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-76cfa42d-1ff7-4124-942a-089562170c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:17.954621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:17.989635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05af409-fd83-4ae8-af9c-8bec23f207fb admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:18.000634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce698a76-1745-4b68-8db0-903abe9714ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:18.001500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce698a76-1745-4b68-8db0-903abe9714ae service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.001867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce698a76-1745-4b68-8db0-903abe9714ae service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.002251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce698a76-1745-4b68-8db0-903abe9714ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.037668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 806/1614] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:18.039293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeda7d7-0eda-40a0-b4f6-d5038bc2210e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.039586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeda7d7-0eda-40a0-b4f6-d5038bc2210e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.040717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeda7d7-0eda-40a0-b4f6-d5038bc2210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.099411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-bdeda7d7-0eda-40a0-b4f6-d5038bc2210e None None] MFA Rules not processed for user `ad318bf4dd64445496bd87d0bd64f3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:18.101489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-bdeda7d7-0eda-40a0-b4f6-d5038bc2210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:18.115535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce698a76-1745-4b68-8db0-903abe9714ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:18.118897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce698a76-1745-4b68-8db0-903abe9714ae service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:18.145862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 809/1615] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:18.147428 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce698a76-1745-4b68-8db0-903abe9714ae service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:18.228321 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 807/1616] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:18.228321 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeda7d7-0eda-40a0-b4f6-d5038bc2210e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:18.263240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeda7d7-0eda-40a0-b4f6-d5038bc2210e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:18.272178 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce698a76-1745-4b68-8db0-903abe9714ae service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:18.277195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4aef0-563a-4391-9019-73f3ae59322d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:18.278328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4aef0-563a-4391-9019-73f3ae59322d service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.278751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4aef0-563a-4391-9019-73f3ae59322d service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.279175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4aef0-563a-4391-9019-73f3ae59322d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.280138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49ab16-e057-4463-8d5b-fd5aac60927f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:18.284048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49ab16-e057-4463-8d5b-fd5aac60927f admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.284392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49ab16-e057-4463-8d5b-fd5aac60927f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.284796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49ab16-e057-4463-8d5b-fd5aac60927f admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.285515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49ab16-e057-4463-8d5b-fd5aac60927f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:18.296478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49ab16-e057-4463-8d5b-fd5aac60927f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:18.328412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 808/1617] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:18 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:18.330255 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791f709-7e50-4516-b213-1ab306294dfa None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.330548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791f709-7e50-4516-b213-1ab306294dfa None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.330909 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791f709-7e50-4516-b213-1ab306294dfa None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.333326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 809/1618] 10.222.0.36 () {54 vars in 962 bytes} [Thu Aug 18 14:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:51:18.334859 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7333b2-a488-4009-a163-8114ed4635a0 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.335206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7333b2-a488-4009-a163-8114ed4635a0 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.335541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7333b2-a488-4009-a163-8114ed4635a0 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.341296 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:18.343854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 810/1619] 10.222.0.36 () {54 vars in 962 bytes} [Thu Aug 18 14:51:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:51:18.345698 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7333b2-a488-4009-a163-8114ed4635a0 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:18.369626 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7333b2-a488-4009-a163-8114ed4635a0 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:18.380345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4aef0-563a-4391-9019-73f3ae59322d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:18.382458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4aef0-563a-4391-9019-73f3ae59322d service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:18.392300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d794f6-6a76-4ab5-b64e-4d8027cf0fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:18.393046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d794f6-6a76-4ab5-b64e-4d8027cf0fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.393547 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d794f6-6a76-4ab5-b64e-4d8027cf0fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.394042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d794f6-6a76-4ab5-b64e-4d8027cf0fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.397564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d794f6-6a76-4ab5-b64e-4d8027cf0fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:18.399232 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d794f6-6a76-4ab5-b64e-4d8027cf0fd9 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:18.406636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 810/1620] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:18.409262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628eafa-4550-4deb-a190-230d242015b0 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.409692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628eafa-4550-4deb-a190-230d242015b0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.410193 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628eafa-4550-4deb-a190-230d242015b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.427482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 811/1621] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:18.431566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02d248-a2f9-457e-8a87-3ef293a92176 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.434272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02d248-a2f9-457e-8a87-3ef293a92176 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.434617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02d248-a2f9-457e-8a87-3ef293a92176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.464810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-9628eafa-4550-4deb-a190-230d242015b0 None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:18.466794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9628eafa-4550-4deb-a190-230d242015b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:18.468567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3c02d248-a2f9-457e-8a87-3ef293a92176 None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:18.469973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3c02d248-a2f9-457e-8a87-3ef293a92176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:18.576382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 811/1622] 10.222.0.36 () {58 vars in 1059 bytes} [Thu Aug 18 14:51:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:18.577475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628eafa-4550-4deb-a190-230d242015b0 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:18.591050 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628eafa-4550-4deb-a190-230d242015b0 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:18.599704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 812/1623] 10.222.0.36 () {58 vars in 1059 bytes} [Thu Aug 18 14:51:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:18.601463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02d248-a2f9-457e-8a87-3ef293a92176 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:18.602589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc2be7-327b-4841-be0a-552d6748d83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:18.603284 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc2be7-327b-4841-be0a-552d6748d83c admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.603629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc2be7-327b-4841-be0a-552d6748d83c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.603984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc2be7-327b-4841-be0a-552d6748d83c admin admin] PATH_INFO: `/v3/projects/03f4349bdacf46118078913950595b03/users/3c42c9aee1a249279e0856272d876548/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.608980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:18.621334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02d248-a2f9-457e-8a87-3ef293a92176 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:18.641197 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c33b28-c131-4b5a-8d9f-558cc535d105 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:18.647169 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc2be7-327b-4841-be0a-552d6748d83c admin admin] RBAC: Authorizing `identity:create_grant(project_id=03f4349bdacf46118078913950595b03, user_id=3c42c9aee1a249279e0856272d876548, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:18.655455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc2be7-327b-4841-be0a-552d6748d83c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:18.670504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c33b28-c131-4b5a-8d9f-558cc535d105 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.672498 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c33b28-c131-4b5a-8d9f-558cc535d105 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.674463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c33b28-c131-4b5a-8d9f-558cc535d105 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.687909 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 812/1624] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:18 2022] PUT /identity/v3/projects/03f4349bdacf46118078913950595b03/users/3c42c9aee1a249279e0856272d876548/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:18.691334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc2be7-327b-4841-be0a-552d6748d83c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:18.719558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c33b28-c131-4b5a-8d9f-558cc535d105 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:18.724184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c33b28-c131-4b5a-8d9f-558cc535d105 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:18.753890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 813/1625] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:18.757346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c33b28-c131-4b5a-8d9f-558cc535d105 service nova] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:18.781077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c33b28-c131-4b5a-8d9f-558cc535d105 service nova] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:18.875813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc2be7-327b-4841-be0a-552d6748d83c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:18.884824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9928804-b587-42e6-a43b-e2f4c0b7be5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:18.886226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9928804-b587-42e6-a43b-e2f4c0b7be5b service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.886511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9928804-b587-42e6-a43b-e2f4c0b7be5b service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.886786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9928804-b587-42e6-a43b-e2f4c0b7be5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.935204 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:18.969236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f44827-520a-4f6c-9d84-7f8dece5dfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:18.970264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f44827-520a-4f6c-9d84-7f8dece5dfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:18.970682 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f44827-520a-4f6c-9d84-7f8dece5dfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:18.970983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f44827-520a-4f6c-9d84-7f8dece5dfa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:18.972243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f44827-520a-4f6c-9d84-7f8dece5dfa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:18.975241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f44827-520a-4f6c-9d84-7f8dece5dfa6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:18.976011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9928804-b587-42e6-a43b-e2f4c0b7be5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:18.978399 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9928804-b587-42e6-a43b-e2f4c0b7be5b service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:18.991084 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 813/1626] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:19.016326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 814/1627] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:19.020590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9928804-b587-42e6-a43b-e2f4c0b7be5b service nova] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:19.035992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9928804-b587-42e6-a43b-e2f4c0b7be5b service nova] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:19.049841 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb28c3-4445-4024-b806-abb55bf8a3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:19.050873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb28c3-4445-4024-b806-abb55bf8a3ab admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:19.050873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb28c3-4445-4024-b806-abb55bf8a3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:19.051504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb28c3-4445-4024-b806-abb55bf8a3ab admin admin] PATH_INFO: `/v3/projects/03f4349bdacf46118078913950595b03/users/3c42c9aee1a249279e0856272d876548/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:19.056245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb28c3-4445-4024-b806-abb55bf8a3ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=03f4349bdacf46118078913950595b03, user_id=3c42c9aee1a249279e0856272d876548, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:19.061354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb28c3-4445-4024-b806-abb55bf8a3ab admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:19.101691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 814/1628] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:19 2022] PUT /identity/v3/projects/03f4349bdacf46118078913950595b03/users/3c42c9aee1a249279e0856272d876548/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:51:19.113192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07443929-1116-42cc-8850-2117969e48fd None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:19.113565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07443929-1116-42cc-8850-2117969e48fd None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:19.113834 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07443929-1116-42cc-8850-2117969e48fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:19.225533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-07443929-1116-42cc-8850-2117969e48fd None None] MFA Rules not processed for user `3c42c9aee1a249279e0856272d876548`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:19.226819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-07443929-1116-42cc-8850-2117969e48fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:19.417589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b697d-f7c5-4a81-b70c-6c97bb764507 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:19.418130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b697d-f7c5-4a81-b70c-6c97bb764507 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:19.418130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b697d-f7c5-4a81-b70c-6c97bb764507 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:19.419476 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 815/1629] 10.222.0.36 () {54 vars in 912 bytes} [Thu Aug 18 14:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:51:19.430840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17ccf8-e925-40e5-ae78-b1ed44496064 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:19.435782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17ccf8-e925-40e5-ae78-b1ed44496064 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:19.435782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17ccf8-e925-40e5-ae78-b1ed44496064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:19.469467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-9b17ccf8-e925-40e5-ae78-b1ed44496064 None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:19.474088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9b17ccf8-e925-40e5-ae78-b1ed44496064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:19.497734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 815/1630] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:19.497734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bdbeb-2c46-49c5-af96-3c4e4067b464 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:19.497734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bdbeb-2c46-49c5-af96-3c4e4067b464 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:19.498453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bdbeb-2c46-49c5-af96-3c4e4067b464 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:19.504392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 816/1631] 10.222.0.36 () {54 vars in 912 bytes} [Thu Aug 18 14:51:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:51:19.520323 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574f607-329a-4947-80a6-d1f4688f9d1b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:19.520999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574f607-329a-4947-80a6-d1f4688f9d1b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:19.521300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574f607-329a-4947-80a6-d1f4688f9d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:19.570104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-0574f607-329a-4947-80a6-d1f4688f9d1b None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:19.574800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0574f607-329a-4947-80a6-d1f4688f9d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:19.643280 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 816/1632] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:51:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:51:19.644332 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17ccf8-e925-40e5-ae78-b1ed44496064 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:19.744716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 817/1633] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:51:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:51:19.746678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574f607-329a-4947-80a6-d1f4688f9d1b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:19.755422 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:19.761593 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17ccf8-e925-40e5-ae78-b1ed44496064 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:19.766360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d75b5b-f399-444a-a759-0a0c76b826f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:19.767438 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d75b5b-f399-444a-a759-0a0c76b826f8 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:19.767898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d75b5b-f399-444a-a759-0a0c76b826f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:19.768679 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d75b5b-f399-444a-a759-0a0c76b826f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:19.783175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d75b5b-f399-444a-a759-0a0c76b826f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:19.785850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d75b5b-f399-444a-a759-0a0c76b826f8 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:19.803762 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 817/1634] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:19.859648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574f607-329a-4947-80a6-d1f4688f9d1b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:19.878214 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d75b5b-f399-444a-a759-0a0c76b826f8 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:19.882612 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1c4ea-9af1-4999-91ac-b5ec35ce9c83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:19.883294 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1c4ea-9af1-4999-91ac-b5ec35ce9c83 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:19.883571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1c4ea-9af1-4999-91ac-b5ec35ce9c83 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:19.883870 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1c4ea-9af1-4999-91ac-b5ec35ce9c83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:19.899317 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d75b5b-f399-444a-a759-0a0c76b826f8 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:19.910623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f27495-e921-443d-bc5e-a676a209cee7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:19.911803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f27495-e921-443d-bc5e-a676a209cee7 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:19.912232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f27495-e921-443d-bc5e-a676a209cee7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:19.912949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f27495-e921-443d-bc5e-a676a209cee7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:19.916429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1c4ea-9af1-4999-91ac-b5ec35ce9c83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:19.918108 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1c4ea-9af1-4999-91ac-b5ec35ce9c83 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:19.942435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f27495-e921-443d-bc5e-a676a209cee7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:19.944410 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f27495-e921-443d-bc5e-a676a209cee7 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:19.946680 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 818/1635] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:19.969238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 818/1636] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:20.163970 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1c4ea-9af1-4999-91ac-b5ec35ce9c83 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:20.179499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1c4ea-9af1-4999-91ac-b5ec35ce9c83 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:20.189084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0215b-1f88-4e0a-abc6-2b0cb01126c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:20.189719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0215b-1f88-4e0a-abc6-2b0cb01126c5 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:20.190001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0215b-1f88-4e0a-abc6-2b0cb01126c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:20.190278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0215b-1f88-4e0a-abc6-2b0cb01126c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:20.359839 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0215b-1f88-4e0a-abc6-2b0cb01126c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:20.361717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0215b-1f88-4e0a-abc6-2b0cb01126c5 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:20.383741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 819/1637] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:20.592763 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f27495-e921-443d-bc5e-a676a209cee7 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:20.778317 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:20.853811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f27495-e921-443d-bc5e-a676a209cee7 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:20.867146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:20.869738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:20.880452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:20.891232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] PATH_INFO: `/v3/users/5f0c269f8bfd426b9eb18e011168040b` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:20.895480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0c269f8bfd426b9eb18e011168040b)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:20.900792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:21.047803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0c269f8bfd426b9eb18e011168040b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:21.068386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0c269f8bfd426b9eb18e011168040b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:21.071985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0c269f8bfd426b9eb18e011168040b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:21.075649 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0c269f8bfd426b9eb18e011168040b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:21.183900 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0c269f8bfd426b9eb18e011168040b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:21.215359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 819/1638] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:51:20 2022] DELETE /identity/v3/users/5f0c269f8bfd426b9eb18e011168040b => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:21.219128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080bf47-ad1b-40da-8298-41288c7f309f None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:21.220315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080bf47-ad1b-40da-8298-41288c7f309f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:21.220791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080bf47-ad1b-40da-8298-41288c7f309f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:21.259537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4080bf47-ad1b-40da-8298-41288c7f309f None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:21.263334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4080bf47-ad1b-40da-8298-41288c7f309f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:21.443973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 820/1639] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:51:21.462887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:21.464504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:21.605819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae7913-6398-4221-b81d-4242caa2c481 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:21.615285 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e208c-56c6-4dc1-85eb-0af70bde997e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:21.616744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e208c-56c6-4dc1-85eb-0af70bde997e service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:21.619560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e208c-56c6-4dc1-85eb-0af70bde997e service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:21.620640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e208c-56c6-4dc1-85eb-0af70bde997e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:21.660170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e208c-56c6-4dc1-85eb-0af70bde997e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:21.661889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e208c-56c6-4dc1-85eb-0af70bde997e service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:21.717749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 820/1640] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:23.342837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080bf47-ad1b-40da-8298-41288c7f309f None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:23.345083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4080bf47-ad1b-40da-8298-41288c7f309f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:23.366847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080bf47-ad1b-40da-8298-41288c7f309f None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:23.379515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4739b-c317-4a25-bd1c-7234d70855af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:23.381588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4739b-c317-4a25-bd1c-7234d70855af admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:23.384150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4739b-c317-4a25-bd1c-7234d70855af admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:23.384644 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4739b-c317-4a25-bd1c-7234d70855af admin admin] PATH_INFO: `/v3/projects/7e523324eb6745e0a8b6f2c147c91312` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:23.387774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4739b-c317-4a25-bd1c-7234d70855af admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e523324eb6745e0a8b6f2c147c91312)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:23.392891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4739b-c317-4a25-bd1c-7234d70855af admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:23.542508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:23.564320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-1fb4739b-c317-4a25-bd1c-7234d70855af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e523324eb6745e0a8b6f2c147c91312', 'request_id': 'req-1fb4739b-c317-4a25-bd1c-7234d70855af'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:23.581109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-1fb4739b-c317-4a25-bd1c-7234d70855af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e523324eb6745e0a8b6f2c147c91312', 'request_id': 'req-1fb4739b-c317-4a25-bd1c-7234d70855af'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:23.602908 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-1fb4739b-c317-4a25-bd1c-7234d70855af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e523324eb6745e0a8b6f2c147c91312', 'request_id': 'req-1fb4739b-c317-4a25-bd1c-7234d70855af'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:23.643045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-1fb4739b-c317-4a25-bd1c-7234d70855af admin admin] The token cache is being invalidate because project 7e523324eb6745e0a8b6f2c147c91312 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:51:23.643625 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-1fb4739b-c317-4a25-bd1c-7234d70855af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:23.647382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-1fb4739b-c317-4a25-bd1c-7234d70855af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:23.685173 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fd7ad-d8b2-43b3-9cea-4f23bda3d9a0 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:23.685723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 821/1641] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:51:23 2022] DELETE /identity/v3/projects/7e523324eb6745e0a8b6f2c147c91312 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:51:23.685794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fd7ad-d8b2-43b3-9cea-4f23bda3d9a0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:23.685794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fd7ad-d8b2-43b3-9cea-4f23bda3d9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:23.742662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-c77fd7ad-d8b2-43b3-9cea-4f23bda3d9a0 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:23.743972 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c77fd7ad-d8b2-43b3-9cea-4f23bda3d9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:23.908544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:23.943422 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 821/1642] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:51:23.951669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f382a4-ad29-4aac-8760-79e5a534dee3 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:23.951980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f382a4-ad29-4aac-8760-79e5a534dee3 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:23.952257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f382a4-ad29-4aac-8760-79e5a534dee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:23.991848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-12f382a4-ad29-4aac-8760-79e5a534dee3 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:23.994299 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-12f382a4-ad29-4aac-8760-79e5a534dee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:24.020429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 822/1643] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:24.026662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fd7ad-d8b2-43b3-9cea-4f23bda3d9a0 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:24.056699 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fd7ad-d8b2-43b3-9cea-4f23bda3d9a0 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:24.079188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee211c-56c7-4406-a2c2-23a5391b17a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:24.080159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee211c-56c7-4406-a2c2-23a5391b17a1 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:24.080663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee211c-56c7-4406-a2c2-23a5391b17a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:24.081135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee211c-56c7-4406-a2c2-23a5391b17a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:24.082307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee211c-56c7-4406-a2c2-23a5391b17a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:24.082775 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee211c-56c7-4406-a2c2-23a5391b17a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:24.084859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee211c-56c7-4406-a2c2-23a5391b17a1 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:24.127180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 822/1644] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:24.131942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f382a4-ad29-4aac-8760-79e5a534dee3 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:24.144866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f382a4-ad29-4aac-8760-79e5a534dee3 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:24.162708 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151257bd-77bd-44f1-84ea-ee0361511650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:24.163360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151257bd-77bd-44f1-84ea-ee0361511650 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:24.163621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151257bd-77bd-44f1-84ea-ee0361511650 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:24.163914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151257bd-77bd-44f1-84ea-ee0361511650 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:24.164554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151257bd-77bd-44f1-84ea-ee0361511650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:24.166502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151257bd-77bd-44f1-84ea-ee0361511650 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:24.225841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 823/1645] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:24 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:24.230422 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee211c-56c7-4406-a2c2-23a5391b17a1 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:24.243466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee211c-56c7-4406-a2c2-23a5391b17a1 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:24.411525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fd2e6-4930-4b30-9049-ff6a4e259f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:24.412408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fd2e6-4930-4b30-9049-ff6a4e259f37 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:24.413045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fd2e6-4930-4b30-9049-ff6a4e259f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:24.413422 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fd2e6-4930-4b30-9049-ff6a4e259f37 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:24.414186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4fd2e6-4930-4b30-9049-ff6a4e259f37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:24.418844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4fd2e6-4930-4b30-9049-ff6a4e259f37 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:24.465114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 823/1646] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:24 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:24.477165 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151257bd-77bd-44f1-84ea-ee0361511650 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:24.499756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151257bd-77bd-44f1-84ea-ee0361511650 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:24.513190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56b26e-ef30-4c3e-9e13-76cf445ec08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:24.514619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56b26e-ef30-4c3e-9e13-76cf445ec08b admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:24.515986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56b26e-ef30-4c3e-9e13-76cf445ec08b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:24.516410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56b26e-ef30-4c3e-9e13-76cf445ec08b admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:24.517428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56b26e-ef30-4c3e-9e13-76cf445ec08b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:24.528779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56b26e-ef30-4c3e-9e13-76cf445ec08b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:24.557076 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 824/1647] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:24.562250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fd2e6-4930-4b30-9049-ff6a4e259f37 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:24.603403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fd2e6-4930-4b30-9049-ff6a4e259f37 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:24.640009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961f9ae-e8e2-4107-9e64-dd171e152f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:24.640820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961f9ae-e8e2-4107-9e64-dd171e152f7b admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:24.641259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961f9ae-e8e2-4107-9e64-dd171e152f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:24.641724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961f9ae-e8e2-4107-9e64-dd171e152f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:24.642882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961f9ae-e8e2-4107-9e64-dd171e152f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:24.648798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961f9ae-e8e2-4107-9e64-dd171e152f7b admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:24.704623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 824/1648] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:24.706676 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56b26e-ef30-4c3e-9e13-76cf445ec08b admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:24.723905 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56b26e-ef30-4c3e-9e13-76cf445ec08b admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:24.736403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd07cbc-94b4-452a-91cc-1d5b57692321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:24.738107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd07cbc-94b4-452a-91cc-1d5b57692321 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:24.738388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd07cbc-94b4-452a-91cc-1d5b57692321 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:24.739957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd07cbc-94b4-452a-91cc-1d5b57692321 admin admin] PATH_INFO: `/v3/projects/9b70406d789b43f68b62674ba1ca071f/users/4005d98ba3df4989b9a255126b17e3b2/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:24.777140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd07cbc-94b4-452a-91cc-1d5b57692321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b70406d789b43f68b62674ba1ca071f, user_id=4005d98ba3df4989b9a255126b17e3b2, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:24.780499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd07cbc-94b4-452a-91cc-1d5b57692321 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:24.808408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 825/1649] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:24 2022] PUT /identity/v3/projects/9b70406d789b43f68b62674ba1ca071f/users/4005d98ba3df4989b9a255126b17e3b2/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:51:24.820517 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b162f-4e39-4fa7-93f2-44ef6209f128 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:24.822100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b162f-4e39-4fa7-93f2-44ef6209f128 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:24.823747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b162f-4e39-4fa7-93f2-44ef6209f128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:24.897496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-9a6b162f-4e39-4fa7-93f2-44ef6209f128 None None] MFA Rules not processed for user `4005d98ba3df4989b9a255126b17e3b2`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:24.898969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9a6b162f-4e39-4fa7-93f2-44ef6209f128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:25.023906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 825/1650] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:25.025524 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd07cbc-94b4-452a-91cc-1d5b57692321 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:25.139368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd07cbc-94b4-452a-91cc-1d5b57692321 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:25.147823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3eed2-e631-4e1b-9125-e5bdb98c46f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:25.148908 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3eed2-e631-4e1b-9125-e5bdb98c46f3 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:25.148908 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3eed2-e631-4e1b-9125-e5bdb98c46f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:25.149351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3eed2-e631-4e1b-9125-e5bdb98c46f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:25.150407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3eed2-e631-4e1b-9125-e5bdb98c46f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:25.153853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3eed2-e631-4e1b-9125-e5bdb98c46f3 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:25.220930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 826/1651] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:25 2022] POST /identity/v3/projects => generated 371 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:51:25.222089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b162f-4e39-4fa7-93f2-44ef6209f128 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:25.260522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b162f-4e39-4fa7-93f2-44ef6209f128 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:25.335537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:25.353040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9aa900-7c93-404f-b41e-87e5ce4fcf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:25.354026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9aa900-7c93-404f-b41e-87e5ce4fcf36 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:25.355161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9aa900-7c93-404f-b41e-87e5ce4fcf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:25.355768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9aa900-7c93-404f-b41e-87e5ce4fcf36 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:25.356533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9aa900-7c93-404f-b41e-87e5ce4fcf36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:25.372976 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9aa900-7c93-404f-b41e-87e5ce4fcf36 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:25.447672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 826/1652] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:25 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:25.455795 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3eed2-e631-4e1b-9125-e5bdb98c46f3 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:25.473872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3eed2-e631-4e1b-9125-e5bdb98c46f3 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:25.480508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebf7f4-0165-4f54-bc35-a0ce8b03d825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:25.496104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebf7f4-0165-4f54-bc35-a0ce8b03d825 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:25.497521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebf7f4-0165-4f54-bc35-a0ce8b03d825 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:25.497915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebf7f4-0165-4f54-bc35-a0ce8b03d825 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:25.501858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebf7f4-0165-4f54-bc35-a0ce8b03d825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:25.509468 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebf7f4-0165-4f54-bc35-a0ce8b03d825 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:25.546092 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9aa900-7c93-404f-b41e-87e5ce4fcf36 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:25.552419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 827/1653] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:25.569047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9aa900-7c93-404f-b41e-87e5ce4fcf36 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:25.593359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17b1b3-a96e-4e95-90f4-2d836f24e95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:25.594057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17b1b3-a96e-4e95-90f4-2d836f24e95f admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:25.594337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17b1b3-a96e-4e95-90f4-2d836f24e95f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:25.594735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17b1b3-a96e-4e95-90f4-2d836f24e95f admin admin] PATH_INFO: `/v3/projects/fd5679f66ce3433e972e5bfae20d0aad/users/7f2717cb9eb24fc59c4c404b9ffc01de/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:25.641710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17b1b3-a96e-4e95-90f4-2d836f24e95f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd5679f66ce3433e972e5bfae20d0aad, user_id=7f2717cb9eb24fc59c4c404b9ffc01de, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:25.645505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17b1b3-a96e-4e95-90f4-2d836f24e95f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:25.681253 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebf7f4-0165-4f54-bc35-a0ce8b03d825 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:25.682957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 827/1654] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:25 2022] PUT /identity/v3/projects/fd5679f66ce3433e972e5bfae20d0aad/users/7f2717cb9eb24fc59c4c404b9ffc01de/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:25.702691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebf7f4-0165-4f54-bc35-a0ce8b03d825 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:25.795920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:25.848061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60adfdd-c66f-4356-a6e8-4a11fc98827f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:25.848898 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60adfdd-c66f-4356-a6e8-4a11fc98827f admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:25.849284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60adfdd-c66f-4356-a6e8-4a11fc98827f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:25.849668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60adfdd-c66f-4356-a6e8-4a11fc98827f admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:25.850450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60adfdd-c66f-4356-a6e8-4a11fc98827f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:25.852262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60adfdd-c66f-4356-a6e8-4a11fc98827f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:25.876226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 828/1655] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:51:25.877524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17b1b3-a96e-4e95-90f4-2d836f24e95f admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:25.892287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17b1b3-a96e-4e95-90f4-2d836f24e95f admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:25.897341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7138a-91ef-438f-a030-c05678f6bcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:25.898092 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7138a-91ef-438f-a030-c05678f6bcdd admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:25.898761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7138a-91ef-438f-a030-c05678f6bcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:25.898761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7138a-91ef-438f-a030-c05678f6bcdd admin admin] PATH_INFO: `/v3/projects/fd5679f66ce3433e972e5bfae20d0aad/users/7f2717cb9eb24fc59c4c404b9ffc01de/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:25.902407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7138a-91ef-438f-a030-c05678f6bcdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd5679f66ce3433e972e5bfae20d0aad, user_id=7f2717cb9eb24fc59c4c404b9ffc01de, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:25.905242 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7138a-91ef-438f-a030-c05678f6bcdd admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:25.967124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 828/1656] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:25 2022] PUT /identity/v3/projects/fd5679f66ce3433e972e5bfae20d0aad/users/7f2717cb9eb24fc59c4c404b9ffc01de/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:25.976642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ca79a-70c1-4d1d-aa9f-a8cdd21f3faf None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:25.977567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ca79a-70c1-4d1d-aa9f-a8cdd21f3faf None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:25.978410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ca79a-70c1-4d1d-aa9f-a8cdd21f3faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:26.071819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-7f3ca79a-70c1-4d1d-aa9f-a8cdd21f3faf None None] MFA Rules not processed for user `7f2717cb9eb24fc59c4c404b9ffc01de`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:26.073986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7f3ca79a-70c1-4d1d-aa9f-a8cdd21f3faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:26.357863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 829/1657] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:26.372469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7138a-91ef-438f-a030-c05678f6bcdd admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:26.378102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-cee7138a-91ef-438f-a030-c05678f6bcdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:26.553958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7138a-91ef-438f-a030-c05678f6bcdd admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:26.582394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab99385-69e2-4ff3-9f8a-83bcef509d50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:26.583032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab99385-69e2-4ff3-9f8a-83bcef509d50 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:26.583309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab99385-69e2-4ff3-9f8a-83bcef509d50 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:26.583594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab99385-69e2-4ff3-9f8a-83bcef509d50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:26.638865 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab99385-69e2-4ff3-9f8a-83bcef509d50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:26.640554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab99385-69e2-4ff3-9f8a-83bcef509d50 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:26.722066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 829/1658] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:27.191835 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232d4c7-5617-4a34-944d-352225ee56e0 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:27.192351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232d4c7-5617-4a34-944d-352225ee56e0 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:27.192351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232d4c7-5617-4a34-944d-352225ee56e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:27.292008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-7232d4c7-5617-4a34-944d-352225ee56e0 None None] MFA Rules not processed for user `1469a960bd9241008cfb112a28bcd5d3`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:27.293281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7232d4c7-5617-4a34-944d-352225ee56e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:27.483188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 830/1659] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:51:27.498700 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975f7d6-c12a-4f72-8cdf-c56f7d6c6f68 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:27.499719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975f7d6-c12a-4f72-8cdf-c56f7d6c6f68 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:27.500252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975f7d6-c12a-4f72-8cdf-c56f7d6c6f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:27.540715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-4975f7d6-c12a-4f72-8cdf-c56f7d6c6f68 None None] MFA Rules not processed for user `1469a960bd9241008cfb112a28bcd5d3`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:27.544297 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4975f7d6-c12a-4f72-8cdf-c56f7d6c6f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:27.569016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 830/1660] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:27.579922 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232d4c7-5617-4a34-944d-352225ee56e0 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:27.583463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7232d4c7-5617-4a34-944d-352225ee56e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:27.755890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:27.770226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975f7d6-c12a-4f72-8cdf-c56f7d6c6f68 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:27.864026 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232d4c7-5617-4a34-944d-352225ee56e0 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:27.901340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fd0de-ab44-4be2-9585-2f5286cc3fb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:27.903518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd0de-ab44-4be2-9585-2f5286cc3fb8 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:27.903518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd0de-ab44-4be2-9585-2f5286cc3fb8 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:27.903518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd0de-ab44-4be2-9585-2f5286cc3fb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:27.921450 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975f7d6-c12a-4f72-8cdf-c56f7d6c6f68 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:27.923619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6fd0de-ab44-4be2-9585-2f5286cc3fb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:27.925511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6fd0de-ab44-4be2-9585-2f5286cc3fb8 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:27.936391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:27.937282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:27.939672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:27.939672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] PATH_INFO: `/v3/users/4005d98ba3df4989b9a255126b17e3b2` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:27.941070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4005d98ba3df4989b9a255126b17e3b2)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:27.944130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:27.954063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 831/1661] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:51:28.022935 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:28.026038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4005d98ba3df4989b9a255126b17e3b2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:28.064998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4005d98ba3df4989b9a255126b17e3b2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:28.067023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4005d98ba3df4989b9a255126b17e3b2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:28.068456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4005d98ba3df4989b9a255126b17e3b2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:28.152753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4005d98ba3df4989b9a255126b17e3b2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:28.220634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 831/1662] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:51:27 2022] DELETE /identity/v3/users/4005d98ba3df4989b9a255126b17e3b2 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:51:28.221722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906e8c1-9ca2-46fe-9761-565bc7fe9d26 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:28.221722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906e8c1-9ca2-46fe-9761-565bc7fe9d26 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:28.221722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906e8c1-9ca2-46fe-9761-565bc7fe9d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:28.266411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-5906e8c1-9ca2-46fe-9761-565bc7fe9d26 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:28.269291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5906e8c1-9ca2-46fe-9761-565bc7fe9d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:28.455151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 832/1663] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:28.481477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:28.484602 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:28.638950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13eea2b-cb49-4639-9851-a88cd5f20821 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:28.644210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c894c37-a3b6-4fce-b38e-4a3744e726c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:28.649373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c894c37-a3b6-4fce-b38e-4a3744e726c0 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:28.649875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c894c37-a3b6-4fce-b38e-4a3744e726c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:28.650292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c894c37-a3b6-4fce-b38e-4a3744e726c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:28.663245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c894c37-a3b6-4fce-b38e-4a3744e726c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:28.664965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c894c37-a3b6-4fce-b38e-4a3744e726c0 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:28.691116 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 832/1664] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:29.008519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535450-4ad5-4aae-b097-ab383ab8e450 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:29.008762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535450-4ad5-4aae-b097-ab383ab8e450 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:29.009049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535450-4ad5-4aae-b097-ab383ab8e450 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:29.010480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 833/1665] 10.222.0.36 () {54 vars in 912 bytes} [Thu Aug 18 14:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:51:29.019362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8026a-0989-4e81-8ee1-b666d939b644 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:29.019776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8026a-0989-4e81-8ee1-b666d939b644 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:29.020155 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8026a-0989-4e81-8ee1-b666d939b644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:29.127038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-79a8026a-0989-4e81-8ee1-b666d939b644 None None] MFA Rules not processed for user `00f209d8426a49acaca7e9d00a1b9ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:29.128619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-79a8026a-0989-4e81-8ee1-b666d939b644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:29.175697 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 833/1666] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:51:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:51:29.227195 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f535450-4ad5-4aae-b097-ab383ab8e450 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:29.231023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8f535450-4ad5-4aae-b097-ab383ab8e450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:29.264439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f535450-4ad5-4aae-b097-ab383ab8e450 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:29.275174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8819ae6-87ae-44cf-a02e-26d1940b0476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:29.277631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8819ae6-87ae-44cf-a02e-26d1940b0476 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:29.278255 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8819ae6-87ae-44cf-a02e-26d1940b0476 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:29.279020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8819ae6-87ae-44cf-a02e-26d1940b0476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:29.300479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8819ae6-87ae-44cf-a02e-26d1940b0476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:29.303613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8819ae6-87ae-44cf-a02e-26d1940b0476 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:29.318295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 834/1667] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:29.763665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8026a-0989-4e81-8ee1-b666d939b644 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:29.765543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-79a8026a-0989-4e81-8ee1-b666d939b644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:29.793142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8026a-0989-4e81-8ee1-b666d939b644 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:29.810948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:29.811625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:29.811958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:29.812229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] PATH_INFO: `/v3/projects/9b70406d789b43f68b62674ba1ca071f` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:29.813822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b70406d789b43f68b62674ba1ca071f)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:29.814861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: WARNING py.warnings [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:51:29.814861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: warnings.warn(msg) Aug 18 14:51:29.814861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: Aug 18 14:51:29.818734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:29.986386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b70406d789b43f68b62674ba1ca071f', 'request_id': 'req-0feed1a4-36d0-4273-8741-13e5838ba0a3'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:29.998130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b70406d789b43f68b62674ba1ca071f', 'request_id': 'req-0feed1a4-36d0-4273-8741-13e5838ba0a3'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:30.017670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b70406d789b43f68b62674ba1ca071f', 'request_id': 'req-0feed1a4-36d0-4273-8741-13e5838ba0a3'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:30.056890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] The token cache is being invalidate because project 9b70406d789b43f68b62674ba1ca071f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:51:30.057713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:30.058587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:30.060298 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 834/1668] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:51:29 2022] DELETE /identity/v3/projects/9b70406d789b43f68b62674ba1ca071f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:51:30.064694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8819ae6-87ae-44cf-a02e-26d1940b0476 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:30.084362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8819ae6-87ae-44cf-a02e-26d1940b0476 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:30.188702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff9ff6-05fa-4057-a50f-44f022287934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:30.189806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff9ff6-05fa-4057-a50f-44f022287934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:30.190293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff9ff6-05fa-4057-a50f-44f022287934 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:30.190574 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff9ff6-05fa-4057-a50f-44f022287934 admin admin] PATH_INFO: `/v3/users/7f2717cb9eb24fc59c4c404b9ffc01de` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:30.192015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff9ff6-05fa-4057-a50f-44f022287934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2717cb9eb24fc59c4c404b9ffc01de)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:30.193163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: WARNING py.warnings [None req-26ff9ff6-05fa-4057-a50f-44f022287934 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 14:51:30.193163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: warnings.warn(msg) Aug 18 14:51:30.193163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: Aug 18 14:51:30.194403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff9ff6-05fa-4057-a50f-44f022287934 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:30.375111 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-26ff9ff6-05fa-4057-a50f-44f022287934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2717cb9eb24fc59c4c404b9ffc01de'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:30.421101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-26ff9ff6-05fa-4057-a50f-44f022287934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2717cb9eb24fc59c4c404b9ffc01de'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:30.423288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-26ff9ff6-05fa-4057-a50f-44f022287934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2717cb9eb24fc59c4c404b9ffc01de'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:30.448441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:30.473353 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-26ff9ff6-05fa-4057-a50f-44f022287934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2717cb9eb24fc59c4c404b9ffc01de'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:30.500930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-26ff9ff6-05fa-4057-a50f-44f022287934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2717cb9eb24fc59c4c404b9ffc01de'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:30.518783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 835/1669] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:51:30 2022] DELETE /identity/v3/users/7f2717cb9eb24fc59c4c404b9ffc01de => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:31.534048 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:31.536716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:31.603012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feed1a4-36d0-4273-8741-13e5838ba0a3 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:31.618500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181d2ff-0255-49b7-8888-b08bdc250ae3 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:31.619523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181d2ff-0255-49b7-8888-b08bdc250ae3 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:31.620587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181d2ff-0255-49b7-8888-b08bdc250ae3 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:31.625445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 836/1670] 10.222.0.36 () {54 vars in 961 bytes} [Thu Aug 18 14:51:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 14:51:31.629089 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75b647-4971-4a74-b7ac-892fabfe8f24 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:31.629669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75b647-4971-4a74-b7ac-892fabfe8f24 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:31.630197 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75b647-4971-4a74-b7ac-892fabfe8f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:31.664105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:31.703671 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3c75b647-4971-4a74-b7ac-892fabfe8f24 None None] MFA Rules not processed for user `db581b10d78a4f9eae33fadce0d92b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:31.705451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3c75b647-4971-4a74-b7ac-892fabfe8f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:31.737915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abd1b1-808a-4ade-88e9-671c30e7b2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:31.738586 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abd1b1-808a-4ade-88e9-671c30e7b2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:31.738867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abd1b1-808a-4ade-88e9-671c30e7b2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:31.739138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abd1b1-808a-4ade-88e9-671c30e7b2b6 admin admin] PATH_INFO: `/v3/users/ad318bf4dd64445496bd87d0bd64f3e2` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:31.740449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abd1b1-808a-4ade-88e9-671c30e7b2b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad318bf4dd64445496bd87d0bd64f3e2)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:31.742714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abd1b1-808a-4ade-88e9-671c30e7b2b6 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:31.850387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-72abd1b1-808a-4ade-88e9-671c30e7b2b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad318bf4dd64445496bd87d0bd64f3e2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:31.866345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-72abd1b1-808a-4ade-88e9-671c30e7b2b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad318bf4dd64445496bd87d0bd64f3e2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:31.867460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-72abd1b1-808a-4ade-88e9-671c30e7b2b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad318bf4dd64445496bd87d0bd64f3e2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:31.872555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-72abd1b1-808a-4ade-88e9-671c30e7b2b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad318bf4dd64445496bd87d0bd64f3e2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:31.898096 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-72abd1b1-808a-4ade-88e9-671c30e7b2b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad318bf4dd64445496bd87d0bd64f3e2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:31.924385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 837/1671] 10.222.0.36 () {58 vars in 1058 bytes} [Thu Aug 18 14:51:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:31.924938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 835/1672] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:51:31 2022] DELETE /identity/v3/users/ad318bf4dd64445496bd87d0bd64f3e2 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:31.929992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354f4ef-4914-420c-a366-27f671f02bb9 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:31.930557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354f4ef-4914-420c-a366-27f671f02bb9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:31.930657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75b647-4971-4a74-b7ac-892fabfe8f24 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:31.931090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354f4ef-4914-420c-a366-27f671f02bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:31.954628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75b647-4971-4a74-b7ac-892fabfe8f24 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:31.985806 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-b354f4ef-4914-420c-a366-27f671f02bb9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:31.987500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b354f4ef-4914-420c-a366-27f671f02bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:32.046032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d77de-6435-4ed8-8257-88167e4d4040 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:32.047477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d77de-6435-4ed8-8257-88167e4d4040 service placement] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:32.047864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d77de-6435-4ed8-8257-88167e4d4040 service placement] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:32.048206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d77de-6435-4ed8-8257-88167e4d4040 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:32.050777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-bf8d77de-6435-4ed8-8257-88167e4d4040 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:32.067379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d77de-6435-4ed8-8257-88167e4d4040 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:32.069038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d77de-6435-4ed8-8257-88167e4d4040 service placement] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:32.084235 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 838/1673] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:32.177069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 836/1674] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:51:32.367477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d77de-6435-4ed8-8257-88167e4d4040 service placement] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:32.369869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-bf8d77de-6435-4ed8-8257-88167e4d4040 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:32.586893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d77de-6435-4ed8-8257-88167e4d4040 service placement] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:32.598818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903660f-34aa-4bac-b71f-b65bfaa322ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:32.601129 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903660f-34aa-4bac-b71f-b65bfaa322ea service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:32.601434 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903660f-34aa-4bac-b71f-b65bfaa322ea service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:32.601716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903660f-34aa-4bac-b71f-b65bfaa322ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:32.633109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903660f-34aa-4bac-b71f-b65bfaa322ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:32.644637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903660f-34aa-4bac-b71f-b65bfaa322ea service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:32.698768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 839/1675] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:32.912876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354f4ef-4914-420c-a366-27f671f02bb9 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:32.915033 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b354f4ef-4914-420c-a366-27f671f02bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:32.946886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354f4ef-4914-420c-a366-27f671f02bb9 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:32.972490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:32.973168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:32.973445 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:32.973928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] PATH_INFO: `/v3/projects/fd5679f66ce3433e972e5bfae20d0aad` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:32.976300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd5679f66ce3433e972e5bfae20d0aad)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:32.978255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:33.174999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:33.189671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd5679f66ce3433e972e5bfae20d0aad', 'request_id': 'req-a80ecae3-fc1e-4ed3-94f0-846cecb649df'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:33.236752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd5679f66ce3433e972e5bfae20d0aad', 'request_id': 'req-a80ecae3-fc1e-4ed3-94f0-846cecb649df'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:33.269229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd5679f66ce3433e972e5bfae20d0aad', 'request_id': 'req-a80ecae3-fc1e-4ed3-94f0-846cecb649df'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:33.347302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] The token cache is being invalidate because project fd5679f66ce3433e972e5bfae20d0aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:51:33.348527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:33.354903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:33.409901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 837/1676] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:51:32 2022] DELETE /identity/v3/projects/fd5679f66ce3433e972e5bfae20d0aad => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:33.417365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090707e6-1265-4d12-a849-6ee5e27b123b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:33.418052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090707e6-1265-4d12-a849-6ee5e27b123b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:33.418654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090707e6-1265-4d12-a849-6ee5e27b123b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:33.554448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-090707e6-1265-4d12-a849-6ee5e27b123b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:33.574789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-090707e6-1265-4d12-a849-6ee5e27b123b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:33.606534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:33.669941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:33.672842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:33.913449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 840/1677] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:33.924461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d0b29-c8c1-42c8-a6cb-834e0af469b0 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:33.924817 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d0b29-c8c1-42c8-a6cb-834e0af469b0 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:33.927590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d0b29-c8c1-42c8-a6cb-834e0af469b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:34.011117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ecae3-fc1e-4ed3-94f0-846cecb649df admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:34.017945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-c94d0b29-c8c1-42c8-a6cb-834e0af469b0 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:34.026903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c94d0b29-c8c1-42c8-a6cb-834e0af469b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:34.031176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690b44c-2331-4ccb-b24a-02481e6b6837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:34.035419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690b44c-2331-4ccb-b24a-02481e6b6837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:34.036080 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690b44c-2331-4ccb-b24a-02481e6b6837 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:34.037180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690b44c-2331-4ccb-b24a-02481e6b6837 admin admin] PATH_INFO: `/v3/users/e3792e07c7ec49b78b3e202858a5b84f` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:34.038949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690b44c-2331-4ccb-b24a-02481e6b6837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3792e07c7ec49b78b3e202858a5b84f)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:34.041486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690b44c-2331-4ccb-b24a-02481e6b6837 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:34.065040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 841/1678] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:34.085906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d0b29-c8c1-42c8-a6cb-834e0af469b0 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:34.110215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d0b29-c8c1-42c8-a6cb-834e0af469b0 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:34.129314 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb6555-66b2-401a-9302-ff05c4f405c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:34.130695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb6555-66b2-401a-9302-ff05c4f405c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:34.131579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb6555-66b2-401a-9302-ff05c4f405c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:34.136651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb6555-66b2-401a-9302-ff05c4f405c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:34.137657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb6555-66b2-401a-9302-ff05c4f405c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:34.138477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb6555-66b2-401a-9302-ff05c4f405c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:34.141629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb6555-66b2-401a-9302-ff05c4f405c3 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:34.195297 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 842/1679] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:34.196788 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb6555-66b2-401a-9302-ff05c4f405c3 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:34.208255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8690b44c-2331-4ccb-b24a-02481e6b6837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3792e07c7ec49b78b3e202858a5b84f'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:34.231390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb6555-66b2-401a-9302-ff05c4f405c3 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:34.266210 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d00100-b5f1-46a9-a5a2-0899e9067b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:34.269329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d00100-b5f1-46a9-a5a2-0899e9067b95 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:34.271248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d00100-b5f1-46a9-a5a2-0899e9067b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:34.272100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d00100-b5f1-46a9-a5a2-0899e9067b95 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:34.273271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d00100-b5f1-46a9-a5a2-0899e9067b95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:34.278430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d00100-b5f1-46a9-a5a2-0899e9067b95 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:34.287923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8690b44c-2331-4ccb-b24a-02481e6b6837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3792e07c7ec49b78b3e202858a5b84f'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:34.290963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8690b44c-2331-4ccb-b24a-02481e6b6837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3792e07c7ec49b78b3e202858a5b84f'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:34.291898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8690b44c-2331-4ccb-b24a-02481e6b6837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3792e07c7ec49b78b3e202858a5b84f'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:34.329407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8690b44c-2331-4ccb-b24a-02481e6b6837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3792e07c7ec49b78b3e202858a5b84f'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:34.376654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 843/1680] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:34 2022] POST /identity/v3/projects => generated 373 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:34.377934 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d00100-b5f1-46a9-a5a2-0899e9067b95 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:34.378618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 838/1681] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:51:33 2022] DELETE /identity/v3/users/e3792e07c7ec49b78b3e202858a5b84f => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:51:34.384130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea262e7-9f90-4759-9319-be44c5f1d7f0 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:34.384664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea262e7-9f90-4759-9319-be44c5f1d7f0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:34.385215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea262e7-9f90-4759-9319-be44c5f1d7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:34.385364 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-01d00100-b5f1-46a9-a5a2-0899e9067b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:34.455748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-9ea262e7-9f90-4759-9319-be44c5f1d7f0 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:34.463584 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9ea262e7-9f90-4759-9319-be44c5f1d7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:34.579508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d00100-b5f1-46a9-a5a2-0899e9067b95 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:34.610278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:34.672256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 839/1682] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:51:34.742257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea262e7-9f90-4759-9319-be44c5f1d7f0 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:34.745686 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9ea262e7-9f90-4759-9319-be44c5f1d7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:34.830962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66863912-7dc1-4b4d-bde5-62af9fea54e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:34.831613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66863912-7dc1-4b4d-bde5-62af9fea54e7 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:34.834061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66863912-7dc1-4b4d-bde5-62af9fea54e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:34.835341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66863912-7dc1-4b4d-bde5-62af9fea54e7 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:34.842183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66863912-7dc1-4b4d-bde5-62af9fea54e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:34.851333 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66863912-7dc1-4b4d-bde5-62af9fea54e7 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:34.957414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 844/1683] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:34 2022] POST /identity/v3/users => generated 287 bytes in 579 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:34.961368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66863912-7dc1-4b4d-bde5-62af9fea54e7 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:34.986202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:35.001204 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66863912-7dc1-4b4d-bde5-62af9fea54e7 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:35.022306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24084eef-bf87-4172-a21c-aee119c23d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:35.024741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24084eef-bf87-4172-a21c-aee119c23d17 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:35.025029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24084eef-bf87-4172-a21c-aee119c23d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:35.027212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24084eef-bf87-4172-a21c-aee119c23d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:35.028318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea262e7-9f90-4759-9319-be44c5f1d7f0 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:35.029117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24084eef-bf87-4172-a21c-aee119c23d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:35.033745 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24084eef-bf87-4172-a21c-aee119c23d17 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:35.045768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2effc94-22f3-4422-9f10-db8acc772800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:35.047487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2effc94-22f3-4422-9f10-db8acc772800 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:35.048704 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2effc94-22f3-4422-9f10-db8acc772800 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:35.050108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2effc94-22f3-4422-9f10-db8acc772800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:35.068867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 845/1684] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:35.074078 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24084eef-bf87-4172-a21c-aee119c23d17 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:35.078185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2effc94-22f3-4422-9f10-db8acc772800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:35.081232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2effc94-22f3-4422-9f10-db8acc772800 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:35.097092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24084eef-bf87-4172-a21c-aee119c23d17 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:35.106643 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 840/1685] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:51:35.109279 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e97d91-ef01-492b-9363-014c80521308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:35.112226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e97d91-ef01-492b-9363-014c80521308 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:35.112226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e97d91-ef01-492b-9363-014c80521308 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:35.118854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e97d91-ef01-492b-9363-014c80521308 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:35.120554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e97d91-ef01-492b-9363-014c80521308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:35.122345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e97d91-ef01-492b-9363-014c80521308 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:35.152677 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2effc94-22f3-4422-9f10-db8acc772800 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:35.153292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 846/1686] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:35.164479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2effc94-22f3-4422-9f10-db8acc772800 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:35.183313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d396a1-8532-49c3-bad3-df7e4a4ed400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:35.184199 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d396a1-8532-49c3-bad3-df7e4a4ed400 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:35.184583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d396a1-8532-49c3-bad3-df7e4a4ed400 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:35.185305 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d396a1-8532-49c3-bad3-df7e4a4ed400 admin admin] PATH_INFO: `/v3/projects/d21ddf0d090342588bedf10f6f1bd142/users/b6b06275a0c94b8187fb1485154f57b8/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:35.223038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d396a1-8532-49c3-bad3-df7e4a4ed400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d21ddf0d090342588bedf10f6f1bd142, user_id=b6b06275a0c94b8187fb1485154f57b8, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:35.232531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d396a1-8532-49c3-bad3-df7e4a4ed400 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:35.254753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 841/1687] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:35 2022] PUT /identity/v3/projects/d21ddf0d090342588bedf10f6f1bd142/users/b6b06275a0c94b8187fb1485154f57b8/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:35.267315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59b0be-7518-4be1-8cec-eb25c660690f None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:35.267592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59b0be-7518-4be1-8cec-eb25c660690f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:35.267920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59b0be-7518-4be1-8cec-eb25c660690f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:35.303733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-dc59b0be-7518-4be1-8cec-eb25c660690f None None] MFA Rules not processed for user `b6b06275a0c94b8187fb1485154f57b8`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:35.305556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-dc59b0be-7518-4be1-8cec-eb25c660690f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:35.428213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d396a1-8532-49c3-bad3-df7e4a4ed400 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:35.429924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 847/1688] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:35.602641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d396a1-8532-49c3-bad3-df7e4a4ed400 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:35.618024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92c801-acce-473e-b901-2918ccfe1182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:35.618692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92c801-acce-473e-b901-2918ccfe1182 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:35.618965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92c801-acce-473e-b901-2918ccfe1182 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:35.619229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92c801-acce-473e-b901-2918ccfe1182 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:35.620122 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92c801-acce-473e-b901-2918ccfe1182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:35.639262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92c801-acce-473e-b901-2918ccfe1182 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:35.724741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 842/1689] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:35 2022] POST /identity/v3/projects => generated 377 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:51:35.729737 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59b0be-7518-4be1-8cec-eb25c660690f None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:35.744198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59b0be-7518-4be1-8cec-eb25c660690f None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:35.960738 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12221bb-8a7d-4ee3-9234-55a6fe5ab491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:35.987409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12221bb-8a7d-4ee3-9234-55a6fe5ab491 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:35.987996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12221bb-8a7d-4ee3-9234-55a6fe5ab491 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:35.989192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12221bb-8a7d-4ee3-9234-55a6fe5ab491 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:35.990671 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12221bb-8a7d-4ee3-9234-55a6fe5ab491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:35.998879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12221bb-8a7d-4ee3-9234-55a6fe5ab491 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:36.040800 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92c801-acce-473e-b901-2918ccfe1182 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:36.063827 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 848/1690] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:35 2022] POST /identity/v3/users => generated 295 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:36.071881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92c801-acce-473e-b901-2918ccfe1182 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:36.084239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84733853-0057-4697-a61a-4d4f5a7f2191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:36.085324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84733853-0057-4697-a61a-4d4f5a7f2191 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:36.085709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84733853-0057-4697-a61a-4d4f5a7f2191 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:36.086230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84733853-0057-4697-a61a-4d4f5a7f2191 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:36.088097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84733853-0057-4697-a61a-4d4f5a7f2191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:36.093995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84733853-0057-4697-a61a-4d4f5a7f2191 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:36.119031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12221bb-8a7d-4ee3-9234-55a6fe5ab491 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:36.119354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 843/1691] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:36.144446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12221bb-8a7d-4ee3-9234-55a6fe5ab491 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:36.159716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af255322-6d74-4031-a13f-b841446a6b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:36.160612 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af255322-6d74-4031-a13f-b841446a6b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:36.161221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af255322-6d74-4031-a13f-b841446a6b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:36.164352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af255322-6d74-4031-a13f-b841446a6b04 admin admin] PATH_INFO: `/v3/projects/bdcdc1361778439da3e1cec7964483fa/users/c9a46a7ab9554eb284fb8d8f9d3da202/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:36.196777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af255322-6d74-4031-a13f-b841446a6b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdcdc1361778439da3e1cec7964483fa, user_id=c9a46a7ab9554eb284fb8d8f9d3da202, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:36.203139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af255322-6d74-4031-a13f-b841446a6b04 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:36.238000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 849/1692] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:36 2022] PUT /identity/v3/projects/bdcdc1361778439da3e1cec7964483fa/users/c9a46a7ab9554eb284fb8d8f9d3da202/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:36.238993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84733853-0057-4697-a61a-4d4f5a7f2191 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:36.253923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84733853-0057-4697-a61a-4d4f5a7f2191 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:36.323369 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af255322-6d74-4031-a13f-b841446a6b04 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:36.337870 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-af255322-6d74-4031-a13f-b841446a6b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:36.420525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:36.485680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009dd940-d5ec-4b06-9fa1-8bc636b88808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:36.487109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dd940-d5ec-4b06-9fa1-8bc636b88808 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:36.487608 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dd940-d5ec-4b06-9fa1-8bc636b88808 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:36.488120 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dd940-d5ec-4b06-9fa1-8bc636b88808 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:36.489704 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009dd940-d5ec-4b06-9fa1-8bc636b88808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:36.512798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009dd940-d5ec-4b06-9fa1-8bc636b88808 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:36.518411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:36.541717 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 844/1693] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:36.544282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68d83d-74a1-4506-8a7d-853ba291dc12 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:36.544560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68d83d-74a1-4506-8a7d-853ba291dc12 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:36.548672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68d83d-74a1-4506-8a7d-853ba291dc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:36.570349 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af255322-6d74-4031-a13f-b841446a6b04 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:36.583078 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:36.584093 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:36.584458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:36.584900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] PATH_INFO: `/v3/projects/aded651ca7894034b7495f2fc3d58f51` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:36.586619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aded651ca7894034b7495f2fc3d58f51)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:36.590716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:36.623666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-6f68d83d-74a1-4506-8a7d-853ba291dc12 None None] MFA Rules not processed for user `36dbc938bcd741f2a2b2013b6aa6f008`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:36.630391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6f68d83d-74a1-4506-8a7d-853ba291dc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:36.745115 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aded651ca7894034b7495f2fc3d58f51', 'request_id': 'req-fa737cab-d962-4e96-ba15-84509fd22dd6'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:36.780298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aded651ca7894034b7495f2fc3d58f51', 'request_id': 'req-fa737cab-d962-4e96-ba15-84509fd22dd6'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:36.805396 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aded651ca7894034b7495f2fc3d58f51', 'request_id': 'req-fa737cab-d962-4e96-ba15-84509fd22dd6'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:36.836379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 845/1694] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:36.838184 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68d83d-74a1-4506-8a7d-853ba291dc12 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:36.842074 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] The token cache is being invalidate because project aded651ca7894034b7495f2fc3d58f51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:51:36.844131 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:36.846407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:36.853778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 850/1695] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:51:36 2022] DELETE /identity/v3/projects/aded651ca7894034b7495f2fc3d58f51 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:36.855359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:36.859396 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:36.889112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68d83d-74a1-4506-8a7d-853ba291dc12 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:37.073341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1406bc7-4170-4a34-9a6d-8e1db829f662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:37.075044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1406bc7-4170-4a34-9a6d-8e1db829f662 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:37.075475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1406bc7-4170-4a34-9a6d-8e1db829f662 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:37.075949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1406bc7-4170-4a34-9a6d-8e1db829f662 admin admin] PATH_INFO: `/v3/projects/bdcdc1361778439da3e1cec7964483fa/users/c9a46a7ab9554eb284fb8d8f9d3da202/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:37.087130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1406bc7-4170-4a34-9a6d-8e1db829f662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdcdc1361778439da3e1cec7964483fa, user_id=c9a46a7ab9554eb284fb8d8f9d3da202, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:37.107543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1406bc7-4170-4a34-9a6d-8e1db829f662 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:37.116882 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa737cab-d962-4e96-ba15-84509fd22dd6 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:37.141677 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:37.142346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:37.142619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:37.142911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] PATH_INFO: `/v3/projects/224e153875984ef291dda96266f56f81` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:37.146780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=224e153875984ef291dda96266f56f81)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:37.148826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:37.215250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 846/1696] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:36 2022] PUT /identity/v3/projects/bdcdc1361778439da3e1cec7964483fa/users/c9a46a7ab9554eb284fb8d8f9d3da202/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:37.237953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4869405-05c2-4f7b-a818-dd0d3f9a4e33 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:37.238276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4869405-05c2-4f7b-a818-dd0d3f9a4e33 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:37.238520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4869405-05c2-4f7b-a818-dd0d3f9a4e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:37.358082 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '224e153875984ef291dda96266f56f81', 'request_id': 'req-492bc7e5-ec1b-42a0-8463-328e053ddb6b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:37.378462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '224e153875984ef291dda96266f56f81', 'request_id': 'req-492bc7e5-ec1b-42a0-8463-328e053ddb6b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:37.404880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '224e153875984ef291dda96266f56f81', 'request_id': 'req-492bc7e5-ec1b-42a0-8463-328e053ddb6b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:37.428146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:37.445650 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f4869405-05c2-4f7b-a818-dd0d3f9a4e33 None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:37.446914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f4869405-05c2-4f7b-a818-dd0d3f9a4e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:37.447475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] The token cache is being invalidate because project 224e153875984ef291dda96266f56f81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:51:37.448793 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:37.452097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:37.460436 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 851/1697] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:51:36 2022] DELETE /identity/v3/projects/224e153875984ef291dda96266f56f81 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:37.461721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:37.470673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:37.546231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:37.617844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 847/1698] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:51:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:37.619169 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6c409-d792-4288-981c-7eb611d5053d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:37.619465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6c409-d792-4288-981c-7eb611d5053d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:37.619809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6c409-d792-4288-981c-7eb611d5053d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:37.665048 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-4ab6c409-d792-4288-981c-7eb611d5053d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:37.666223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4ab6c409-d792-4288-981c-7eb611d5053d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:37.728343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bc7e5-ec1b-42a0-8463-328e053ddb6b admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:37.735440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cb284-6bc7-4ad8-940f-20810ad77f89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:37.736306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cb284-6bc7-4ad8-940f-20810ad77f89 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:37.736732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cb284-6bc7-4ad8-940f-20810ad77f89 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:37.737094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cb284-6bc7-4ad8-940f-20810ad77f89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:37.741445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d57cb284-6bc7-4ad8-940f-20810ad77f89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:37.862628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 848/1699] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:37.864502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d27f4-8b3d-4af5-aff5-69632197edff None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:37.865011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d27f4-8b3d-4af5-aff5-69632197edff None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:37.865484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d27f4-8b3d-4af5-aff5-69632197edff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:37.926388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cb284-6bc7-4ad8-940f-20810ad77f89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:37.934978 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cb284-6bc7-4ad8-940f-20810ad77f89 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:37.958675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-586d27f4-8b3d-4af5-aff5-69632197edff None None] MFA Rules not processed for user `c9a46a7ab9554eb284fb8d8f9d3da202`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:37.960686 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-586d27f4-8b3d-4af5-aff5-69632197edff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:37.965210 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 852/1700] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:37.966769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092becbf-3de1-4ccb-a12b-758b063dfd70 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:37.967063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092becbf-3de1-4ccb-a12b-758b063dfd70 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:37.967349 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092becbf-3de1-4ccb-a12b-758b063dfd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:38.067286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-092becbf-3de1-4ccb-a12b-758b063dfd70 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:38.068672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-092becbf-3de1-4ccb-a12b-758b063dfd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:38.105471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 853/1701] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:38.111802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011413a2-9540-400f-a58f-d65cb4fb6a09 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:38.112097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011413a2-9540-400f-a58f-d65cb4fb6a09 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:38.112385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011413a2-9540-400f-a58f-d65cb4fb6a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:38.154361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 849/1702] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:38.158786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d27f4-8b3d-4af5-aff5-69632197edff None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:38.161670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-586d27f4-8b3d-4af5-aff5-69632197edff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:38.190708 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-011413a2-9540-400f-a58f-d65cb4fb6a09 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:38.191994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-011413a2-9540-400f-a58f-d65cb4fb6a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:38.223102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 854/1703] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:38.224439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f682b-aabf-42a4-9040-31fd3c7220da None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:38.224726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f682b-aabf-42a4-9040-31fd3c7220da None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:38.225001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f682b-aabf-42a4-9040-31fd3c7220da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:38.274173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-906f682b-aabf-42a4-9040-31fd3c7220da None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:38.275472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-906f682b-aabf-42a4-9040-31fd3c7220da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:38.298966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 855/1704] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:38.300083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f682b-aabf-42a4-9040-31fd3c7220da None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:38.303646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-906f682b-aabf-42a4-9040-31fd3c7220da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:38.337575 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:38.405293 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d27f4-8b3d-4af5-aff5-69632197edff None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:38.422984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cbc43-1386-46d3-beec-d872337d0605 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:38.422984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cbc43-1386-46d3-beec-d872337d0605 service swift] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:38.423921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cbc43-1386-46d3-beec-d872337d0605 service swift] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:38.424612 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cbc43-1386-46d3-beec-d872337d0605 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:38.444339 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cbc43-1386-46d3-beec-d872337d0605 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:38.447133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cbc43-1386-46d3-beec-d872337d0605 service swift] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:38.471489 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 850/1705] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:51:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:51:38.481338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cbc43-1386-46d3-beec-d872337d0605 service swift] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:38.507529 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cbc43-1386-46d3-beec-d872337d0605 service swift] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:38.516968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e8d43-9558-4541-ab71-723d80d44dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:38.517596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e8d43-9558-4541-ab71-723d80d44dea admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:38.517873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e8d43-9558-4541-ab71-723d80d44dea admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:38.518144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e8d43-9558-4541-ab71-723d80d44dea admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:38.518886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e8d43-9558-4541-ab71-723d80d44dea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:38.523617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e8d43-9558-4541-ab71-723d80d44dea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:38.525397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e8d43-9558-4541-ab71-723d80d44dea admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:38.557594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f682b-aabf-42a4-9040-31fd3c7220da None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:38.566174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 851/1706] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:38.567262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e8d43-9558-4541-ab71-723d80d44dea admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:38.576143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce1e2f-50dd-41d4-8ca8-6681d711e9c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:38.577202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce1e2f-50dd-41d4-8ca8-6681d711e9c6 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:38.577775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce1e2f-50dd-41d4-8ca8-6681d711e9c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:38.578295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce1e2f-50dd-41d4-8ca8-6681d711e9c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:38.604275 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-90ce1e2f-50dd-41d4-8ca8-6681d711e9c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:38.606409 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e8d43-9558-4541-ab71-723d80d44dea admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:38.644299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995aa41-2faf-4501-a4ee-2e4092b0b7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:38.644807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995aa41-2faf-4501-a4ee-2e4092b0b7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:38.645088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995aa41-2faf-4501-a4ee-2e4092b0b7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:38.645413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995aa41-2faf-4501-a4ee-2e4092b0b7c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:38.646325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995aa41-2faf-4501-a4ee-2e4092b0b7c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:38.653254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995aa41-2faf-4501-a4ee-2e4092b0b7c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:38.655614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995aa41-2faf-4501-a4ee-2e4092b0b7c9 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:38.705353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 852/1707] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:38.707742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af220117-5829-48dc-8e6c-96cde1361155 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:38.707742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af220117-5829-48dc-8e6c-96cde1361155 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:38.707742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af220117-5829-48dc-8e6c-96cde1361155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:38.797116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:38.864751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-af220117-5829-48dc-8e6c-96cde1361155 None None] MFA Rules not processed for user `4d96208474754b2381d769d807689bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:38.867272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-af220117-5829-48dc-8e6c-96cde1361155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:38.919941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce1e2f-50dd-41d4-8ca8-6681d711e9c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:38.921427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce1e2f-50dd-41d4-8ca8-6681d711e9c6 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:38.955665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 856/1708] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:38.960009 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce1e2f-50dd-41d4-8ca8-6681d711e9c6 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:39.011122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce1e2f-50dd-41d4-8ca8-6681d711e9c6 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:39.024533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2d77e-ab55-42e1-8383-820ab4304b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:39.027467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2d77e-ab55-42e1-8383-820ab4304b83 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:39.027966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2d77e-ab55-42e1-8383-820ab4304b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:39.028495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2d77e-ab55-42e1-8383-820ab4304b83 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:39.033456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 853/1709] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:39.034554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af220117-5829-48dc-8e6c-96cde1361155 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:39.038694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2d77e-ab55-42e1-8383-820ab4304b83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:39.043982 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2d77e-ab55-42e1-8383-820ab4304b83 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:39.070428 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af220117-5829-48dc-8e6c-96cde1361155 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:39.091111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d7a91-f3eb-438f-a655-5319627f2a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:39.093398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d7a91-f3eb-438f-a655-5319627f2a3e admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:39.093872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d7a91-f3eb-438f-a655-5319627f2a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:39.094328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d7a91-f3eb-438f-a655-5319627f2a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:39.107794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d7a91-f3eb-438f-a655-5319627f2a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:39.107794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d7a91-f3eb-438f-a655-5319627f2a3e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:39.197635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 857/1710] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:38 2022] POST /identity/v3/projects => generated 381 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:39.198629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2d77e-ab55-42e1-8383-820ab4304b83 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:39.248621 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 854/1711] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:39 2022] POST /identity/v3/projects => generated 365 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:39.258256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc292fbe-d761-4bc8-87cd-453077b02cf4 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:39.258544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc292fbe-d761-4bc8-87cd-453077b02cf4 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:39.266673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc292fbe-d761-4bc8-87cd-453077b02cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:39.457405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-fc292fbe-d761-4bc8-87cd-453077b02cf4 None None] MFA Rules not processed for user `09b055bb15ce4b2ab4276b0090e16122`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:39.471076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-fc292fbe-d761-4bc8-87cd-453077b02cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:39.571453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2d77e-ab55-42e1-8383-820ab4304b83 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:39.801552 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 855/1712] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 552 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:39.803044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc292fbe-d761-4bc8-87cd-453077b02cf4 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:39.817559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:39.985984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15542626-c84d-4796-ba1d-3d36d94c359c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:39.986607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542626-c84d-4796-ba1d-3d36d94c359c service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:39.987177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542626-c84d-4796-ba1d-3d36d94c359c service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:39.987819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542626-c84d-4796-ba1d-3d36d94c359c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:40.064495 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc292fbe-d761-4bc8-87cd-453077b02cf4 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:40.080309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed02e85-ad80-4340-aea5-a0f1ca2eda56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:40.082202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed02e85-ad80-4340-aea5-a0f1ca2eda56 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:40.082507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed02e85-ad80-4340-aea5-a0f1ca2eda56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:40.083963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed02e85-ad80-4340-aea5-a0f1ca2eda56 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:40.085345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed02e85-ad80-4340-aea5-a0f1ca2eda56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:40.094790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed02e85-ad80-4340-aea5-a0f1ca2eda56 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:40.157330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 856/1713] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:39 2022] POST /identity/v3/users => generated 291 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:40.161144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed02e85-ad80-4340-aea5-a0f1ca2eda56 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:40.161422 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:40.204673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed02e85-ad80-4340-aea5-a0f1ca2eda56 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:40.217386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442f51b-bb09-41e2-8c06-b62fba92fc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:40.218547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442f51b-bb09-41e2-8c06-b62fba92fc19 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:40.219660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442f51b-bb09-41e2-8c06-b62fba92fc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:40.220300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442f51b-bb09-41e2-8c06-b62fba92fc19 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:40.220914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15542626-c84d-4796-ba1d-3d36d94c359c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:40.221836 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442f51b-bb09-41e2-8c06-b62fba92fc19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:40.225128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442f51b-bb09-41e2-8c06-b62fba92fc19 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:40.230151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15542626-c84d-4796-ba1d-3d36d94c359c service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:40.264948 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 858/1714] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 1067 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:40.276956 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 857/1715] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:40 2022] POST /identity/v3/users => generated 283 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:40.278599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442f51b-bb09-41e2-8c06-b62fba92fc19 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:40.281789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877f64b-7557-4ba4-bec4-04536301f3ac None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:40.282290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877f64b-7557-4ba4-bec4-04536301f3ac None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:40.282806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877f64b-7557-4ba4-bec4-04536301f3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:40.284657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f442f51b-bb09-41e2-8c06-b62fba92fc19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:40.349543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442f51b-bb09-41e2-8c06-b62fba92fc19 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:40.382217 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b48e6f-aa69-4b7f-98e1-33390d8e7056 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:40.382728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b48e6f-aa69-4b7f-98e1-33390d8e7056 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:40.383162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b48e6f-aa69-4b7f-98e1-33390d8e7056 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:40.383543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b48e6f-aa69-4b7f-98e1-33390d8e7056 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:40.397010 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-d877f64b-7557-4ba4-bec4-04536301f3ac None None] MFA Rules not processed for user `00f209d8426a49acaca7e9d00a1b9ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:40.402892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b48e6f-aa69-4b7f-98e1-33390d8e7056 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:40.405120 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b48e6f-aa69-4b7f-98e1-33390d8e7056 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:40.419460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 858/1716] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:40.420558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b48e6f-aa69-4b7f-98e1-33390d8e7056 service nova] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:40.437583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d877f64b-7557-4ba4-bec4-04536301f3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:40.442842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b48e6f-aa69-4b7f-98e1-33390d8e7056 service nova] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:40.460365 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c1b56-b280-4816-882a-0d828ea65a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:40.461627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c1b56-b280-4816-882a-0d828ea65a5f admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:40.461953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c1b56-b280-4816-882a-0d828ea65a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:40.462562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c1b56-b280-4816-882a-0d828ea65a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:40.464282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c1b56-b280-4816-882a-0d828ea65a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:40.472829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c1b56-b280-4816-882a-0d828ea65a5f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:40.495808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 859/1717] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:40.496901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c1b56-b280-4816-882a-0d828ea65a5f admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:40.520403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c1b56-b280-4816-882a-0d828ea65a5f admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:40.538059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d23507-9f5d-4193-b51f-9ed33b16fa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:40.539149 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d23507-9f5d-4193-b51f-9ed33b16fa41 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:40.539574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d23507-9f5d-4193-b51f-9ed33b16fa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:40.542739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d23507-9f5d-4193-b51f-9ed33b16fa41 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:40.550816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d23507-9f5d-4193-b51f-9ed33b16fa41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:40.553067 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d23507-9f5d-4193-b51f-9ed33b16fa41 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:40.596090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 860/1718] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:40.597165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d23507-9f5d-4193-b51f-9ed33b16fa41 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:40.617985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d23507-9f5d-4193-b51f-9ed33b16fa41 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:40.624963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735bed6-5f84-4817-8a5f-3b6943d9d30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:40.625798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735bed6-5f84-4817-8a5f-3b6943d9d30d admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:40.626309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735bed6-5f84-4817-8a5f-3b6943d9d30d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:40.626777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735bed6-5f84-4817-8a5f-3b6943d9d30d admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:40.627610 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735bed6-5f84-4817-8a5f-3b6943d9d30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:40.630068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735bed6-5f84-4817-8a5f-3b6943d9d30d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:40.658709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 861/1719] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:40.658709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735bed6-5f84-4817-8a5f-3b6943d9d30d admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:40.676676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735bed6-5f84-4817-8a5f-3b6943d9d30d admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:40.685623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12926493-9b55-4b41-8ce2-dc6523b59234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:40.686449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12926493-9b55-4b41-8ce2-dc6523b59234 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:40.687034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12926493-9b55-4b41-8ce2-dc6523b59234 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:40.687548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12926493-9b55-4b41-8ce2-dc6523b59234 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:40.688873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12926493-9b55-4b41-8ce2-dc6523b59234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:40.691170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12926493-9b55-4b41-8ce2-dc6523b59234 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:40.734260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 862/1720] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:40.736455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12926493-9b55-4b41-8ce2-dc6523b59234 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:40.770064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12926493-9b55-4b41-8ce2-dc6523b59234 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:40.783483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b3b9f-3b79-43a9-9e35-3771fe687d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:40.785523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b3b9f-3b79-43a9-9e35-3771fe687d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:40.785905 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b3b9f-3b79-43a9-9e35-3771fe687d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:40.786638 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b3b9f-3b79-43a9-9e35-3771fe687d88 admin admin] PATH_INFO: `/v3/projects/f4ffa19da6de49398c88ba3c2b9d2b43/users/f2b7be9e7fa848ecbb4d3af4ec3af714/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:40.793498 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 859/1721] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Aug 18 14:51:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 528 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:40.801967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877f64b-7557-4ba4-bec4-04536301f3ac None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:40.831406 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b3b9f-3b79-43a9-9e35-3771fe687d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ffa19da6de49398c88ba3c2b9d2b43, user_id=f2b7be9e7fa848ecbb4d3af4ec3af714, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:40.835632 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b3b9f-3b79-43a9-9e35-3771fe687d88 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:40.864121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877f64b-7557-4ba4-bec4-04536301f3ac None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:40.875178 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 863/1722] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:40 2022] PUT /identity/v3/projects/f4ffa19da6de49398c88ba3c2b9d2b43/users/f2b7be9e7fa848ecbb4d3af4ec3af714/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:40.877409 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b3b9f-3b79-43a9-9e35-3771fe687d88 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:40.883219 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b68b3b9f-3b79-43a9-9e35-3771fe687d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:41.070433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81583219-ee36-42ff-8b86-77b647f77d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:41.071414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81583219-ee36-42ff-8b86-77b647f77d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:41.072286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81583219-ee36-42ff-8b86-77b647f77d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:41.072286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81583219-ee36-42ff-8b86-77b647f77d6d admin admin] PATH_INFO: `/v3/projects/67c24225c9964207974f34e4cbee9ec1/users/48cb4944c0124affa78ccb515db2b79e/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:41.105295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b3b9f-3b79-43a9-9e35-3771fe687d88 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:41.114807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81583219-ee36-42ff-8b86-77b647f77d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=67c24225c9964207974f34e4cbee9ec1, user_id=48cb4944c0124affa78ccb515db2b79e, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:41.124484 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81583219-ee36-42ff-8b86-77b647f77d6d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:41.136089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53672014-9344-49a3-b4de-0f6926fbd147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:41.137255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53672014-9344-49a3-b4de-0f6926fbd147 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:41.140435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53672014-9344-49a3-b4de-0f6926fbd147 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:41.141877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53672014-9344-49a3-b4de-0f6926fbd147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:41.151805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:41.155957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53672014-9344-49a3-b4de-0f6926fbd147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:41.159897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53672014-9344-49a3-b4de-0f6926fbd147 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:41.169269 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 860/1723] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:40 2022] PUT /identity/v3/projects/67c24225c9964207974f34e4cbee9ec1/users/48cb4944c0124affa78ccb515db2b79e/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:41.170839 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce48a9-06b2-4c07-9928-fcdbc6f4bb47 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:41.171127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce48a9-06b2-4c07-9928-fcdbc6f4bb47 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:41.173905 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce48a9-06b2-4c07-9928-fcdbc6f4bb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:41.196127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 864/1724] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:41.197955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24a0ab-10cf-481e-b5e0-8d386541c25e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:41.198357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24a0ab-10cf-481e-b5e0-8d386541c25e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:41.198750 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24a0ab-10cf-481e-b5e0-8d386541c25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:41.255328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-ddce48a9-06b2-4c07-9928-fcdbc6f4bb47 None None] MFA Rules not processed for user `f2b7be9e7fa848ecbb4d3af4ec3af714`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:41.266908 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ddce48a9-06b2-4c07-9928-fcdbc6f4bb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:41.312425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-3d24a0ab-10cf-481e-b5e0-8d386541c25e None None] MFA Rules not processed for user `48cb4944c0124affa78ccb515db2b79e`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:41.335603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3d24a0ab-10cf-481e-b5e0-8d386541c25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:41.542897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 861/1725] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:41.553265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce48a9-06b2-4c07-9928-fcdbc6f4bb47 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:41.579369 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:41.765725 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 865/1726] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 565 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:41.774136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24a0ab-10cf-481e-b5e0-8d386541c25e None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:41.963573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce48a9-06b2-4c07-9928-fcdbc6f4bb47 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:41.987393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a51f9-6fb4-47da-be1e-67065d7f2387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:41.989362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a51f9-6fb4-47da-be1e-67065d7f2387 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:41.992297 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a51f9-6fb4-47da-be1e-67065d7f2387 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:41.993755 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a51f9-6fb4-47da-be1e-67065d7f2387 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:41.996839 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a51f9-6fb4-47da-be1e-67065d7f2387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:42.005614 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a51f9-6fb4-47da-be1e-67065d7f2387 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:42.134563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 862/1727] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:41 2022] POST /identity/v3/projects => generated 381 bytes in 585 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:42.138716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a51f9-6fb4-47da-be1e-67065d7f2387 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:42.180611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24a0ab-10cf-481e-b5e0-8d386541c25e None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:42.382669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a51f9-6fb4-47da-be1e-67065d7f2387 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:42.403615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bc140-d1b3-4aa7-b142-15e9c5ea70bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:42.405931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bc140-d1b3-4aa7-b142-15e9c5ea70bc admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:42.406051 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae4180-9fe3-4046-a415-7f123ccce366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:42.406561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bc140-d1b3-4aa7-b142-15e9c5ea70bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:42.406662 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae4180-9fe3-4046-a415-7f123ccce366 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:42.406941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae4180-9fe3-4046-a415-7f123ccce366 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:42.407219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae4180-9fe3-4046-a415-7f123ccce366 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:42.407537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bc140-d1b3-4aa7-b142-15e9c5ea70bc admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:42.408594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bc140-d1b3-4aa7-b142-15e9c5ea70bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:42.411289 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bc140-d1b3-4aa7-b142-15e9c5ea70bc admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:42.414872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae4180-9fe3-4046-a415-7f123ccce366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:42.420562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae4180-9fe3-4046-a415-7f123ccce366 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:42.489314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 866/1728] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:41 2022] POST /identity/v3/projects => generated 363 bytes in 720 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:42.497467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 863/1729] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:42 2022] POST /identity/v3/users => generated 297 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:51:42.500148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bc140-d1b3-4aa7-b142-15e9c5ea70bc admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:42.502111 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae4180-9fe3-4046-a415-7f123ccce366 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:42.544123 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae4180-9fe3-4046-a415-7f123ccce366 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:42.552400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:42.744113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bc140-d1b3-4aa7-b142-15e9c5ea70bc admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:42.768736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c58ce5-a678-4487-bc22-f6649d292227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:42.770984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c58ce5-a678-4487-bc22-f6649d292227 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:42.771177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336da803-d716-415a-88b0-ddc5c4da5a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:42.771920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336da803-d716-415a-88b0-ddc5c4da5a09 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:42.772238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336da803-d716-415a-88b0-ddc5c4da5a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:42.772725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336da803-d716-415a-88b0-ddc5c4da5a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:42.773602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336da803-d716-415a-88b0-ddc5c4da5a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:42.773932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c58ce5-a678-4487-bc22-f6649d292227 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:42.774370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c58ce5-a678-4487-bc22-f6649d292227 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:42.776784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c58ce5-a678-4487-bc22-f6649d292227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:42.778903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336da803-d716-415a-88b0-ddc5c4da5a09 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:42.781685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c58ce5-a678-4487-bc22-f6649d292227 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:42.806019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 864/1730] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:42.812404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336da803-d716-415a-88b0-ddc5c4da5a09 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:42.854301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336da803-d716-415a-88b0-ddc5c4da5a09 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:42.866688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 867/1731] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:42 2022] POST /identity/v3/users => generated 288 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:42.868558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97727d2b-e11c-446a-98c5-20670d75b56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:42.869314 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97727d2b-e11c-446a-98c5-20670d75b56f admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:42.870170 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97727d2b-e11c-446a-98c5-20670d75b56f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:42.870511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97727d2b-e11c-446a-98c5-20670d75b56f admin admin] PATH_INFO: `/v3/projects/d2eba23fd55f45949ccf286476bbb068/users/42e4bdf302394f9a98fa31994c5bc3e4/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:42.871555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c58ce5-a678-4487-bc22-f6649d292227 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:42.895183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c58ce5-a678-4487-bc22-f6649d292227 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:42.904685 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:42.908963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cc454-8ef5-473a-a6fb-f6bfe43b8148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:42.909604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cc454-8ef5-473a-a6fb-f6bfe43b8148 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:42.910295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cc454-8ef5-473a-a6fb-f6bfe43b8148 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:42.910580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cc454-8ef5-473a-a6fb-f6bfe43b8148 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:42.912379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095cc454-8ef5-473a-a6fb-f6bfe43b8148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:42.917682 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97727d2b-e11c-446a-98c5-20670d75b56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2eba23fd55f45949ccf286476bbb068, user_id=42e4bdf302394f9a98fa31994c5bc3e4, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:42.920705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97727d2b-e11c-446a-98c5-20670d75b56f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:42.921758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095cc454-8ef5-473a-a6fb-f6bfe43b8148 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:42.954565 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 868/1732] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:42.956712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cc454-8ef5-473a-a6fb-f6bfe43b8148 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:42.962491 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 865/1733] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:42 2022] PUT /identity/v3/projects/d2eba23fd55f45949ccf286476bbb068/users/42e4bdf302394f9a98fa31994c5bc3e4/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:51:42.973848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97727d2b-e11c-446a-98c5-20670d75b56f admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:43.005691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cc454-8ef5-473a-a6fb-f6bfe43b8148 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:43.203300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97727d2b-e11c-446a-98c5-20670d75b56f admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:43.231519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33357a01-0b9e-4ed5-9d96-2ba1453b7cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:43.232299 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33357a01-0b9e-4ed5-9d96-2ba1453b7cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:43.232588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33357a01-0b9e-4ed5-9d96-2ba1453b7cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:43.232868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33357a01-0b9e-4ed5-9d96-2ba1453b7cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:43.233630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33357a01-0b9e-4ed5-9d96-2ba1453b7cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:43.236434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37edb4-23ae-4c85-a523-ae4b4438ca3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:43.237434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37edb4-23ae-4c85-a523-ae4b4438ca3e admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:43.237706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37edb4-23ae-4c85-a523-ae4b4438ca3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:43.238054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37edb4-23ae-4c85-a523-ae4b4438ca3e admin admin] PATH_INFO: `/v3/projects/a686ea2462114454be6a3620590c0a45/users/2f89629bcdcc476a8e43e8fe64a437bd/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:43.242070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33357a01-0b9e-4ed5-9d96-2ba1453b7cc6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:43.302863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 866/1734] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:43.304712 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33357a01-0b9e-4ed5-9d96-2ba1453b7cc6 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:43.315572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37edb4-23ae-4c85-a523-ae4b4438ca3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a686ea2462114454be6a3620590c0a45, user_id=2f89629bcdcc476a8e43e8fe64a437bd, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:43.322138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37edb4-23ae-4c85-a523-ae4b4438ca3e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:43.352715 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33357a01-0b9e-4ed5-9d96-2ba1453b7cc6 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:43.375653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 869/1735] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:42 2022] PUT /identity/v3/projects/a686ea2462114454be6a3620590c0a45/users/2f89629bcdcc476a8e43e8fe64a437bd/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:43.377825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37edb4-23ae-4c85-a523-ae4b4438ca3e admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:43.524070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a6627-d5fe-485f-806d-36b2a8081b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:43.524837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a6627-d5fe-485f-806d-36b2a8081b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:43.525185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a6627-d5fe-485f-806d-36b2a8081b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:43.528782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a6627-d5fe-485f-806d-36b2a8081b19 admin admin] PATH_INFO: `/v3/projects/d2eba23fd55f45949ccf286476bbb068/users/42e4bdf302394f9a98fa31994c5bc3e4/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:43.533910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a6627-d5fe-485f-806d-36b2a8081b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2eba23fd55f45949ccf286476bbb068, user_id=42e4bdf302394f9a98fa31994c5bc3e4, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:43.544825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a6627-d5fe-485f-806d-36b2a8081b19 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:43.561928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 867/1736] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:43 2022] PUT /identity/v3/projects/d2eba23fd55f45949ccf286476bbb068/users/42e4bdf302394f9a98fa31994c5bc3e4/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:43.575567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848365b-93ba-46ed-8fc4-d9c6fb75de87 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:43.575970 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848365b-93ba-46ed-8fc4-d9c6fb75de87 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:43.576825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848365b-93ba-46ed-8fc4-d9c6fb75de87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:43.694333 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-b848365b-93ba-46ed-8fc4-d9c6fb75de87 None None] MFA Rules not processed for user `42e4bdf302394f9a98fa31994c5bc3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:43.697934 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b848365b-93ba-46ed-8fc4-d9c6fb75de87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:43.746797 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37edb4-23ae-4c85-a523-ae4b4438ca3e admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:43.834192 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:43.895277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 868/1737] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:43.907738 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848365b-93ba-46ed-8fc4-d9c6fb75de87 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:43.915201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b848365b-93ba-46ed-8fc4-d9c6fb75de87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:43.939762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:43.985191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab520793-9126-40b4-ad88-f2b0d27bbd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:43.985838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab520793-9126-40b4-ad88-f2b0d27bbd89 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:43.986122 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab520793-9126-40b4-ad88-f2b0d27bbd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:43.986406 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab520793-9126-40b4-ad88-f2b0d27bbd89 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:43.987076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab520793-9126-40b4-ad88-f2b0d27bbd89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:43.988745 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab520793-9126-40b4-ad88-f2b0d27bbd89 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:44.034828 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 870/1738] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 659 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:44.035855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab520793-9126-40b4-ad88-f2b0d27bbd89 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:44.086159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab520793-9126-40b4-ad88-f2b0d27bbd89 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:44.115914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac61eb8-8397-4d00-9614-fa50953024d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:44.116689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac61eb8-8397-4d00-9614-fa50953024d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:44.116970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac61eb8-8397-4d00-9614-fa50953024d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:44.117244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac61eb8-8397-4d00-9614-fa50953024d1 admin admin] PATH_INFO: `/v3/projects/a686ea2462114454be6a3620590c0a45/users/2f89629bcdcc476a8e43e8fe64a437bd/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:44.133423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac61eb8-8397-4d00-9614-fa50953024d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a686ea2462114454be6a3620590c0a45, user_id=2f89629bcdcc476a8e43e8fe64a437bd, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:44.145638 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac61eb8-8397-4d00-9614-fa50953024d1 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:44.200849 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 871/1739] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:44 2022] PUT /identity/v3/projects/a686ea2462114454be6a3620590c0a45/users/2f89629bcdcc476a8e43e8fe64a437bd/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:44.202600 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7a57d-a113-4827-ad7c-4a2bcf695972 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:44.202995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7a57d-a113-4827-ad7c-4a2bcf695972 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:44.203282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7a57d-a113-4827-ad7c-4a2bcf695972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:44.212032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848365b-93ba-46ed-8fc4-d9c6fb75de87 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:44.344127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-e3d7a57d-a113-4827-ad7c-4a2bcf695972 None None] MFA Rules not processed for user `2f89629bcdcc476a8e43e8fe64a437bd`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:44.345501 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e3d7a57d-a113-4827-ad7c-4a2bcf695972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:44.423491 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa80787-8016-4026-85c7-d9d931695bfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:44.424215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa80787-8016-4026-85c7-d9d931695bfe service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:44.424496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa80787-8016-4026-85c7-d9d931695bfe service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:44.424769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa80787-8016-4026-85c7-d9d931695bfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:44.590834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 872/1740] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:44.636092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa80787-8016-4026-85c7-d9d931695bfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:44.638914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa80787-8016-4026-85c7-d9d931695bfe service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:44.695853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 869/1741] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 789 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:51:44.771832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7a57d-a113-4827-ad7c-4a2bcf695972 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:44.810194 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7a57d-a113-4827-ad7c-4a2bcf695972 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:44.830184 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c605b-a384-4390-8929-ad660b0c9836 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:44.839167 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c605b-a384-4390-8929-ad660b0c9836 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:44.839565 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c605b-a384-4390-8929-ad660b0c9836 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:44.840169 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c605b-a384-4390-8929-ad660b0c9836 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:44.886519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c605b-a384-4390-8929-ad660b0c9836 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:44.893726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c605b-a384-4390-8929-ad660b0c9836 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:44.974668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 873/1742] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:45.044347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa80787-8016-4026-85c7-d9d931695bfe service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:45.062100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa80787-8016-4026-85c7-d9d931695bfe service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:45.081115 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91da4e3-f791-4b94-a677-e6fa795ddf49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:45.081816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91da4e3-f791-4b94-a677-e6fa795ddf49 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:45.082085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91da4e3-f791-4b94-a677-e6fa795ddf49 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:45.082384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91da4e3-f791-4b94-a677-e6fa795ddf49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:45.336517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91da4e3-f791-4b94-a677-e6fa795ddf49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:45.338002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91da4e3-f791-4b94-a677-e6fa795ddf49 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:45.368428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 870/1743] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:45.829993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c605b-a384-4390-8929-ad660b0c9836 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:45.844574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c605b-a384-4390-8929-ad660b0c9836 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:45.868607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8cddb-a5b1-414e-9657-d66033021ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:45.875505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8cddb-a5b1-414e-9657-d66033021ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:45.875851 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8cddb-a5b1-414e-9657-d66033021ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:45.878533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8cddb-a5b1-414e-9657-d66033021ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:46.074454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8cddb-a5b1-414e-9657-d66033021ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:46.078122 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8cddb-a5b1-414e-9657-d66033021ad8 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:46.092102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 874/1744] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:46.212465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df264ac0-1303-45fd-b613-142616cbe023 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:46.212953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df264ac0-1303-45fd-b613-142616cbe023 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:46.212953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df264ac0-1303-45fd-b613-142616cbe023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:46.290681 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-df264ac0-1303-45fd-b613-142616cbe023 None None] MFA Rules not processed for user `36dbc938bcd741f2a2b2013b6aa6f008`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:46.292628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-df264ac0-1303-45fd-b613-142616cbe023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:46.338943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:46.405942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8cddb-a5b1-414e-9657-d66033021ad8 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:46.465130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:46.635527 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 871/1745] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:46.717247 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8cddb-a5b1-414e-9657-d66033021ad8 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:46.735615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34e2fe-cbe3-4ae5-9e21-3a452230470d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:46.736313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34e2fe-cbe3-4ae5-9e21-3a452230470d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:46.736592 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34e2fe-cbe3-4ae5-9e21-3a452230470d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:46.736865 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34e2fe-cbe3-4ae5-9e21-3a452230470d admin admin] PATH_INFO: `/v3/users/48cb4944c0124affa78ccb515db2b79e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:46.744909 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34e2fe-cbe3-4ae5-9e21-3a452230470d admin admin] RBAC: Authorizing `identity:delete_user(user_id=48cb4944c0124affa78ccb515db2b79e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:46.746895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34e2fe-cbe3-4ae5-9e21-3a452230470d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:46.879646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-db34e2fe-cbe3-4ae5-9e21-3a452230470d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48cb4944c0124affa78ccb515db2b79e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:46.904804 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-db34e2fe-cbe3-4ae5-9e21-3a452230470d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48cb4944c0124affa78ccb515db2b79e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:46.906793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-db34e2fe-cbe3-4ae5-9e21-3a452230470d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48cb4944c0124affa78ccb515db2b79e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:46.907838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-db34e2fe-cbe3-4ae5-9e21-3a452230470d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48cb4944c0124affa78ccb515db2b79e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:46.941232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-db34e2fe-cbe3-4ae5-9e21-3a452230470d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48cb4944c0124affa78ccb515db2b79e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:46.969442 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 875/1746] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:51:46 2022] DELETE /identity/v3/users/48cb4944c0124affa78ccb515db2b79e => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:46.975535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0bb0e-44bd-4bb1-8d28-091e1528480b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:46.976432 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0bb0e-44bd-4bb1-8d28-091e1528480b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:46.976858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0bb0e-44bd-4bb1-8d28-091e1528480b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:47.028105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-cdc0bb0e-44bd-4bb1-8d28-091e1528480b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:47.029478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-cdc0bb0e-44bd-4bb1-8d28-091e1528480b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:47.316803 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 872/1747] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:51:47.430213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d0aa8-621a-4110-9ee0-af6a064ed229 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:47.430760 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d0aa8-621a-4110-9ee0-af6a064ed229 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:47.431241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d0aa8-621a-4110-9ee0-af6a064ed229 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:47.434114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 876/1748] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 14:51:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:51:47.488160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0bb0e-44bd-4bb1-8d28-091e1528480b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:47.490845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-cdc0bb0e-44bd-4bb1-8d28-091e1528480b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:47.536651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229594e-8e6e-4bc4-a07f-f317dbe9148a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:47.537042 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229594e-8e6e-4bc4-a07f-f317dbe9148a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:47.537325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229594e-8e6e-4bc4-a07f-f317dbe9148a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:47.591901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-7229594e-8e6e-4bc4-a07f-f317dbe9148a None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:47.593350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7229594e-8e6e-4bc4-a07f-f317dbe9148a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:47.681326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0bb0e-44bd-4bb1-8d28-091e1528480b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:47.696728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b13a29-a197-44ff-8577-2a28eb2b5fa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:47.698196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b13a29-a197-44ff-8577-2a28eb2b5fa5 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:47.702234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b13a29-a197-44ff-8577-2a28eb2b5fa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:47.702651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b13a29-a197-44ff-8577-2a28eb2b5fa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:47.730597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b13a29-a197-44ff-8577-2a28eb2b5fa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:47.732434 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b13a29-a197-44ff-8577-2a28eb2b5fa5 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:47.766003 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 873/1749] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:47.784467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 877/1750] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 14:51:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:51:47.809445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b13a29-a197-44ff-8577-2a28eb2b5fa5 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:47.812386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-62b13a29-a197-44ff-8577-2a28eb2b5fa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:47.877233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b13a29-a197-44ff-8577-2a28eb2b5fa5 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:47.925938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f64c76-ea13-49b1-b27f-6d6d41b2819c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:47.925938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f64c76-ea13-49b1-b27f-6d6d41b2819c service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:47.925938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f64c76-ea13-49b1-b27f-6d6d41b2819c service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:47.925938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f64c76-ea13-49b1-b27f-6d6d41b2819c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:47.960620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f64c76-ea13-49b1-b27f-6d6d41b2819c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:47.962568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f64c76-ea13-49b1-b27f-6d6d41b2819c service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:48.023951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 874/1751] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:48.818482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69b40a-d772-4c9c-adde-c1213f085848 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:48.819520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69b40a-d772-4c9c-adde-c1213f085848 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:48.821174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69b40a-d772-4c9c-adde-c1213f085848 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:48.825456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 878/1752] 10.222.0.36 () {54 vars in 912 bytes} [Thu Aug 18 14:51:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:51:48.845099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65a011-e06c-4a1d-bae6-2cd473981878 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:48.846041 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65a011-e06c-4a1d-bae6-2cd473981878 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:48.846456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65a011-e06c-4a1d-bae6-2cd473981878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:48.861281 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a518ec8-9207-41e6-a43c-642d4c14ca9c None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:48.863575 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a518ec8-9207-41e6-a43c-642d4c14ca9c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:48.864057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a518ec8-9207-41e6-a43c-642d4c14ca9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:48.912403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-2a65a011-e06c-4a1d-bae6-2cd473981878 None None] MFA Rules not processed for user `00f209d8426a49acaca7e9d00a1b9ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:48.916484 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2a65a011-e06c-4a1d-bae6-2cd473981878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:48.944009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-2a518ec8-9207-41e6-a43c-642d4c14ca9c None None] MFA Rules not processed for user `4d96208474754b2381d769d807689bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:48.949691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 875/1753] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:51:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:48.966132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2a518ec8-9207-41e6-a43c-642d4c14ca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:49.001605 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:49.058921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65a011-e06c-4a1d-bae6-2cd473981878 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:49.070989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65a011-e06c-4a1d-bae6-2cd473981878 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:49.088623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b3de9-ec57-48da-94e4-214792c64fd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:49.093158 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b3de9-ec57-48da-94e4-214792c64fd3 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:49.093539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b3de9-ec57-48da-94e4-214792c64fd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:49.096201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b3de9-ec57-48da-94e4-214792c64fd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:49.111264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 879/1754] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:49.144230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b3de9-ec57-48da-94e4-214792c64fd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:49.145697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b3de9-ec57-48da-94e4-214792c64fd3 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:49.177911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 876/1755] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:51:49.659771 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a518ec8-9207-41e6-a43c-642d4c14ca9c None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:49.661915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2a518ec8-9207-41e6-a43c-642d4c14ca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:49.698189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a518ec8-9207-41e6-a43c-642d4c14ca9c None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:49.712763 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:49.713785 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:49.714702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:49.715590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] PATH_INFO: `/v3/projects/67c24225c9964207974f34e4cbee9ec1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:49.718913 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67c24225c9964207974f34e4cbee9ec1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:49.725887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:49.879156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67c24225c9964207974f34e4cbee9ec1', 'request_id': 'req-c0036213-fd09-4ca1-8f8c-9840636e6b20'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:49.896364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67c24225c9964207974f34e4cbee9ec1', 'request_id': 'req-c0036213-fd09-4ca1-8f8c-9840636e6b20'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:49.928202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67c24225c9964207974f34e4cbee9ec1', 'request_id': 'req-c0036213-fd09-4ca1-8f8c-9840636e6b20'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:49.964790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] The token cache is being invalidate because project 67c24225c9964207974f34e4cbee9ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:51:49.971757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:49.974407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:49.984596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 880/1756] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:51:49 2022] DELETE /identity/v3/projects/67c24225c9964207974f34e4cbee9ec1 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:49.997724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:49.997935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9a81c-9f50-4458-b861-b1e0eb7829ca None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:49.998423 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9a81c-9f50-4458-b861-b1e0eb7829ca None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:49.998423 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9a81c-9f50-4458-b861-b1e0eb7829ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:50.000880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:50.046001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3ca9a81c-9f50-4458-b861-b1e0eb7829ca None None] MFA Rules not processed for user `09b055bb15ce4b2ab4276b0090e16122`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:50.047939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3ca9a81c-9f50-4458-b861-b1e0eb7829ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:50.100557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0036213-fd09-4ca1-8f8c-9840636e6b20 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:50.108210 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:50.115648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b4a07-ce11-4261-b8c2-9b749885650c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:50.116492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b4a07-ce11-4261-b8c2-9b749885650c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:50.116776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b4a07-ce11-4261-b8c2-9b749885650c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:50.117053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b4a07-ce11-4261-b8c2-9b749885650c admin admin] PATH_INFO: `/v3/users/2f89629bcdcc476a8e43e8fe64a437bd` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:50.120631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b4a07-ce11-4261-b8c2-9b749885650c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f89629bcdcc476a8e43e8fe64a437bd)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:50.122617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b4a07-ce11-4261-b8c2-9b749885650c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:50.184184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 877/1757] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:50.237260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-547b4a07-ce11-4261-b8c2-9b749885650c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f89629bcdcc476a8e43e8fe64a437bd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:50.249304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-547b4a07-ce11-4261-b8c2-9b749885650c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f89629bcdcc476a8e43e8fe64a437bd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:50.250282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-547b4a07-ce11-4261-b8c2-9b749885650c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f89629bcdcc476a8e43e8fe64a437bd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:50.250912 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-547b4a07-ce11-4261-b8c2-9b749885650c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f89629bcdcc476a8e43e8fe64a437bd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:50.252415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:50.317227 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-547b4a07-ce11-4261-b8c2-9b749885650c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f89629bcdcc476a8e43e8fe64a437bd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:50.370676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 881/1758] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:51:49 2022] DELETE /identity/v3/users/2f89629bcdcc476a8e43e8fe64a437bd => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:53.292667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9a81c-9f50-4458-b861-b1e0eb7829ca None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:53.320450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9a81c-9f50-4458-b861-b1e0eb7829ca None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:53.474721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:53.475412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:53.475950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:53.481635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b admin admin] PATH_INFO: `/v3/projects/a686ea2462114454be6a3620590c0a45` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:53.485628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a686ea2462114454be6a3620590c0a45)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:53.492829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:53.652934 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a686ea2462114454be6a3620590c0a45', 'request_id': 'req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:53.661144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a686ea2462114454be6a3620590c0a45', 'request_id': 'req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:53.674136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a686ea2462114454be6a3620590c0a45', 'request_id': 'req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:53.692127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b admin admin] The token cache is being invalidate because project a686ea2462114454be6a3620590c0a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:51:53.692576 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:53.693450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c98b3b9a-eeca-43d3-b7e8-54a33c008f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:51:53.696538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 878/1759] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:51:53 2022] DELETE /identity/v3/projects/a686ea2462114454be6a3620590c0a45 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:53.702159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00256855-f4fc-43ca-80a3-15b6404f1678 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:53.702672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00256855-f4fc-43ca-80a3-15b6404f1678 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:53.703458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00256855-f4fc-43ca-80a3-15b6404f1678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:53.740809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-00256855-f4fc-43ca-80a3-15b6404f1678 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:53.742422 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-00256855-f4fc-43ca-80a3-15b6404f1678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:53.838894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 882/1760] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:53.846630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ede464-4d9d-4cc9-aef0-416b7d761b3b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:53.846931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ede464-4d9d-4cc9-aef0-416b7d761b3b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:53.847172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ede464-4d9d-4cc9-aef0-416b7d761b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:53.868453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-72ede464-4d9d-4cc9-aef0-416b7d761b3b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:53.870924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-72ede464-4d9d-4cc9-aef0-416b7d761b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:53.888943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 879/1761] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:53.890369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00256855-f4fc-43ca-80a3-15b6404f1678 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:53.906180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00256855-f4fc-43ca-80a3-15b6404f1678 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:53.917738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e9f94-69e7-4982-ae37-6b95f94f7202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:53.918710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e9f94-69e7-4982-ae37-6b95f94f7202 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:53.919406 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e9f94-69e7-4982-ae37-6b95f94f7202 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:53.919780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e9f94-69e7-4982-ae37-6b95f94f7202 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:53.920535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e9f94-69e7-4982-ae37-6b95f94f7202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:51:53.920897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e9f94-69e7-4982-ae37-6b95f94f7202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:53.923264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e9f94-69e7-4982-ae37-6b95f94f7202 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:53.955720 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 883/1762] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:51:53.960387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ede464-4d9d-4cc9-aef0-416b7d761b3b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:53.970837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ede464-4d9d-4cc9-aef0-416b7d761b3b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:53.977061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110afd20-1904-44bc-9d06-10dcaffe66c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:53.978095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110afd20-1904-44bc-9d06-10dcaffe66c9 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:53.978385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110afd20-1904-44bc-9d06-10dcaffe66c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:53.979359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110afd20-1904-44bc-9d06-10dcaffe66c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:53.980047 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110afd20-1904-44bc-9d06-10dcaffe66c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:53.982599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110afd20-1904-44bc-9d06-10dcaffe66c9 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:54.030840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e9f94-69e7-4982-ae37-6b95f94f7202 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:54.031305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 880/1763] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:53 2022] POST /identity/v3/projects => generated 399 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:51:54.042458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e9f94-69e7-4982-ae37-6b95f94f7202 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:54.117944 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c5144-65af-4c3e-91e4-51a44e14c472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:54.118688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c5144-65af-4c3e-91e4-51a44e14c472 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:54.118969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c5144-65af-4c3e-91e4-51a44e14c472 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:54.119272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c5144-65af-4c3e-91e4-51a44e14c472 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:54.119973 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c5144-65af-4c3e-91e4-51a44e14c472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:54.122149 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c5144-65af-4c3e-91e4-51a44e14c472 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:54.145373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 884/1764] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:54 2022] POST /identity/v3/users => generated 300 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:51:54.149762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110afd20-1904-44bc-9d06-10dcaffe66c9 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:54.162764 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110afd20-1904-44bc-9d06-10dcaffe66c9 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:54.168562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b78e1c-cfb9-4b09-8de6-0a06227a3e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:54.169363 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b78e1c-cfb9-4b09-8de6-0a06227a3e9a admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:54.169715 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b78e1c-cfb9-4b09-8de6-0a06227a3e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:54.170058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b78e1c-cfb9-4b09-8de6-0a06227a3e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:54.170981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b78e1c-cfb9-4b09-8de6-0a06227a3e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:54.172709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b78e1c-cfb9-4b09-8de6-0a06227a3e9a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:54.184744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 881/1765] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:51:54.188582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c5144-65af-4c3e-91e4-51a44e14c472 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:54.200442 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c5144-65af-4c3e-91e4-51a44e14c472 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:54.206380 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08afc7-b21f-45b8-950e-6d1ae3e05f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:54.207008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08afc7-b21f-45b8-950e-6d1ae3e05f90 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:54.207320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08afc7-b21f-45b8-950e-6d1ae3e05f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:54.207553 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08afc7-b21f-45b8-950e-6d1ae3e05f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:54.208308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08afc7-b21f-45b8-950e-6d1ae3e05f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:54.209958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08afc7-b21f-45b8-950e-6d1ae3e05f90 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:54.225521 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 885/1766] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:54.228155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b78e1c-cfb9-4b09-8de6-0a06227a3e9a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:54.238180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b78e1c-cfb9-4b09-8de6-0a06227a3e9a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:54.243207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e25507-66c1-4ae5-bc8e-973c42b6b381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:54.243935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e25507-66c1-4ae5-bc8e-973c42b6b381 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:54.244309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e25507-66c1-4ae5-bc8e-973c42b6b381 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:54.244589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e25507-66c1-4ae5-bc8e-973c42b6b381 admin admin] PATH_INFO: `/v3/projects/47c39520582149cda6c44ecf57efa9b8/users/cc830fb444c545d388cd171c220472e6/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:54.259563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e25507-66c1-4ae5-bc8e-973c42b6b381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47c39520582149cda6c44ecf57efa9b8, user_id=cc830fb444c545d388cd171c220472e6, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:54.262797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e25507-66c1-4ae5-bc8e-973c42b6b381 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:54.270881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:54.279418 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 882/1767] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:54 2022] PUT /identity/v3/projects/47c39520582149cda6c44ecf57efa9b8/users/cc830fb444c545d388cd171c220472e6/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:54.291006 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ed35b-1162-4987-9665-9c67178629d4 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:54.291309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ed35b-1162-4987-9665-9c67178629d4 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:54.291579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ed35b-1162-4987-9665-9c67178629d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:54.320445 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-525ed35b-1162-4987-9665-9c67178629d4 None None] MFA Rules not processed for user `cc830fb444c545d388cd171c220472e6`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:54.321715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-525ed35b-1162-4987-9665-9c67178629d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:54.352505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:54.409877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 886/1768] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:54 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:54.414991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e25507-66c1-4ae5-bc8e-973c42b6b381 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:54.502818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e25507-66c1-4ae5-bc8e-973c42b6b381 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:54.508668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd668b8d-2064-4d94-a27e-b6e7ea0ebba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:54.509377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd668b8d-2064-4d94-a27e-b6e7ea0ebba6 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:54.509807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd668b8d-2064-4d94-a27e-b6e7ea0ebba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:54.510722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd668b8d-2064-4d94-a27e-b6e7ea0ebba6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:54.511553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd668b8d-2064-4d94-a27e-b6e7ea0ebba6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:54.514319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd668b8d-2064-4d94-a27e-b6e7ea0ebba6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:54.547244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 883/1769] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:51:54 2022] POST /identity/v3/projects => generated 401 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:51:54.552104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ed35b-1162-4987-9665-9c67178629d4 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:54.565677 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ed35b-1162-4987-9665-9c67178629d4 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:54.629118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c256c-c29f-48d8-a368-7ad990049075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:54.629737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c256c-c29f-48d8-a368-7ad990049075 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:54.629993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c256c-c29f-48d8-a368-7ad990049075 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:54.630259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c256c-c29f-48d8-a368-7ad990049075 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:54.630888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c256c-c29f-48d8-a368-7ad990049075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:54.633109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c256c-c29f-48d8-a368-7ad990049075 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:54.664701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 887/1770] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:51:54 2022] POST /identity/v3/users => generated 307 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:51:54.668797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd668b8d-2064-4d94-a27e-b6e7ea0ebba6 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:54.679268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd668b8d-2064-4d94-a27e-b6e7ea0ebba6 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:54.684607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eb2f9-8278-4c12-a6f4-d341dcfa4305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:54.685215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eb2f9-8278-4c12-a6f4-d341dcfa4305 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:54.685518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eb2f9-8278-4c12-a6f4-d341dcfa4305 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:54.685787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eb2f9-8278-4c12-a6f4-d341dcfa4305 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:54.686430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313eb2f9-8278-4c12-a6f4-d341dcfa4305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:54.688141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313eb2f9-8278-4c12-a6f4-d341dcfa4305 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:54.700064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 884/1771] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:54.704351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c256c-c29f-48d8-a368-7ad990049075 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:54.724018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c256c-c29f-48d8-a368-7ad990049075 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:54.730010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da9e3d-6b0b-4d76-8835-4682d9e99ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:54.730691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da9e3d-6b0b-4d76-8835-4682d9e99ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:54.730973 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da9e3d-6b0b-4d76-8835-4682d9e99ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:54.731394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da9e3d-6b0b-4d76-8835-4682d9e99ad5 admin admin] PATH_INFO: `/v3/projects/86e042aa756245d99acf36c161354aed/users/e2cda203984445ea9c1ed21d27ee5646/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:54.747916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da9e3d-6b0b-4d76-8835-4682d9e99ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e042aa756245d99acf36c161354aed, user_id=e2cda203984445ea9c1ed21d27ee5646, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:54.750628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da9e3d-6b0b-4d76-8835-4682d9e99ad5 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:54.766001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 888/1772] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:54 2022] PUT /identity/v3/projects/86e042aa756245d99acf36c161354aed/users/e2cda203984445ea9c1ed21d27ee5646/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:54.770029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eb2f9-8278-4c12-a6f4-d341dcfa4305 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:54.780084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eb2f9-8278-4c12-a6f4-d341dcfa4305 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:54.845118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f12e8-58b0-4d79-992d-877390d4a998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:54.845731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f12e8-58b0-4d79-992d-877390d4a998 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:54.846018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f12e8-58b0-4d79-992d-877390d4a998 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:54.846293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f12e8-58b0-4d79-992d-877390d4a998 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:54.846940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f12e8-58b0-4d79-992d-877390d4a998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:54.848580 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f12e8-58b0-4d79-992d-877390d4a998 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:54.859648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 885/1773] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:51:54.863993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da9e3d-6b0b-4d76-8835-4682d9e99ad5 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:54.875838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da9e3d-6b0b-4d76-8835-4682d9e99ad5 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:54.881936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe646b-1ece-4e75-a577-efa4ef2f9255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:54.882668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe646b-1ece-4e75-a577-efa4ef2f9255 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:54.883023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe646b-1ece-4e75-a577-efa4ef2f9255 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:54.883316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe646b-1ece-4e75-a577-efa4ef2f9255 admin admin] PATH_INFO: `/v3/projects/86e042aa756245d99acf36c161354aed/users/e2cda203984445ea9c1ed21d27ee5646/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:54.885342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe646b-1ece-4e75-a577-efa4ef2f9255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e042aa756245d99acf36c161354aed, user_id=e2cda203984445ea9c1ed21d27ee5646, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:54.888042 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe646b-1ece-4e75-a577-efa4ef2f9255 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:54.899504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 889/1774] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:51:54 2022] PUT /identity/v3/projects/86e042aa756245d99acf36c161354aed/users/e2cda203984445ea9c1ed21d27ee5646/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:51:54.909769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1d5ea-0e97-4a13-b265-070d49cea52e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:54.909976 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1d5ea-0e97-4a13-b265-070d49cea52e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:54.910268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1d5ea-0e97-4a13-b265-070d49cea52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:54.939873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-0fc1d5ea-0e97-4a13-b265-070d49cea52e None None] MFA Rules not processed for user `e2cda203984445ea9c1ed21d27ee5646`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:51:54.941087 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0fc1d5ea-0e97-4a13-b265-070d49cea52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:55.000271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:51:55.017036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 886/1775] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:51:54 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:51:55.023064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe646b-1ece-4e75-a577-efa4ef2f9255 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:55.024803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7abe646b-1ece-4e75-a577-efa4ef2f9255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:51:55.081964 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe646b-1ece-4e75-a577-efa4ef2f9255 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:55.085584 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57406c-bec6-4913-8498-50e75e115b84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:55.086186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57406c-bec6-4913-8498-50e75e115b84 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:55.086458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57406c-bec6-4913-8498-50e75e115b84 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:55.086723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57406c-bec6-4913-8498-50e75e115b84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:55.095928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57406c-bec6-4913-8498-50e75e115b84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:55.097344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57406c-bec6-4913-8498-50e75e115b84 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:55.107392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 890/1776] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:55 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:51:55.184021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1d5ea-0e97-4a13-b265-070d49cea52e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:51:55.193846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1d5ea-0e97-4a13-b265-070d49cea52e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:51:55.197461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f380fe-36aa-468a-b234-e25fc665e844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:51:55.198072 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f380fe-36aa-468a-b234-e25fc665e844 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:51:55.198347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f380fe-36aa-468a-b234-e25fc665e844 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:51:55.198621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f380fe-36aa-468a-b234-e25fc665e844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:51:55.259086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f380fe-36aa-468a-b234-e25fc665e844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:51:55.261169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f380fe-36aa-468a-b234-e25fc665e844 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:51:55.269782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 887/1777] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:51:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:05.102532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=84,keystone:INSERT=5 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:05.265672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=31 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:10.769275 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87831ffc-5494-49af-ae90-75e10e42083a None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:10.770145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87831ffc-5494-49af-ae90-75e10e42083a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:10.770586 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87831ffc-5494-49af-ae90-75e10e42083a None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:10.772688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 891/1778] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 14:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:52:10.787602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b784d0-ab08-4357-bdaf-6f7d22cb1977 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:10.789609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b784d0-ab08-4357-bdaf-6f7d22cb1977 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:10.792853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b784d0-ab08-4357-bdaf-6f7d22cb1977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:10.899943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-20b784d0-ab08-4357-bdaf-6f7d22cb1977 None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:10.904122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-20b784d0-ab08-4357-bdaf-6f7d22cb1977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:10.920625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87831ffc-5494-49af-ae90-75e10e42083a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:10.922382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-87831ffc-5494-49af-ae90-75e10e42083a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:11.086209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87831ffc-5494-49af-ae90-75e10e42083a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:11.106706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416f4ef-2586-480c-bd1e-bf86a6277740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:11.110479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 888/1779] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:11.110955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416f4ef-2586-480c-bd1e-bf86a6277740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:11.111271 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416f4ef-2586-480c-bd1e-bf86a6277740 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:11.111547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416f4ef-2586-480c-bd1e-bf86a6277740 admin admin] PATH_INFO: `/v3/users/f2b7be9e7fa848ecbb4d3af4ec3af714` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:11.115100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416f4ef-2586-480c-bd1e-bf86a6277740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2b7be9e7fa848ecbb4d3af4ec3af714)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:11.122262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416f4ef-2586-480c-bd1e-bf86a6277740 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:11.233708 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3416f4ef-2586-480c-bd1e-bf86a6277740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2b7be9e7fa848ecbb4d3af4ec3af714'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:11.263453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3416f4ef-2586-480c-bd1e-bf86a6277740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2b7be9e7fa848ecbb4d3af4ec3af714'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:11.266223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3416f4ef-2586-480c-bd1e-bf86a6277740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2b7be9e7fa848ecbb4d3af4ec3af714'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:11.267885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3416f4ef-2586-480c-bd1e-bf86a6277740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2b7be9e7fa848ecbb4d3af4ec3af714'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:11.305691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3416f4ef-2586-480c-bd1e-bf86a6277740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2b7be9e7fa848ecbb4d3af4ec3af714'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:11.328777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b784d0-ab08-4357-bdaf-6f7d22cb1977 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:11.330608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-20b784d0-ab08-4357-bdaf-6f7d22cb1977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:11.337848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 892/1780] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:10 2022] DELETE /identity/v3/users/f2b7be9e7fa848ecbb4d3af4ec3af714 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:11.361288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0180292-5856-4cba-abdc-096511c9db7d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:11.361630 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0180292-5856-4cba-abdc-096511c9db7d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:11.362360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0180292-5856-4cba-abdc-096511c9db7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:11.369328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b784d0-ab08-4357-bdaf-6f7d22cb1977 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:11.400777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-e0180292-5856-4cba-abdc-096511c9db7d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:11.404268 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e0180292-5856-4cba-abdc-096511c9db7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:11.491986 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 893/1781] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:11.494396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0180292-5856-4cba-abdc-096511c9db7d None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:11.498157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e0180292-5856-4cba-abdc-096511c9db7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:11.517620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0180292-5856-4cba-abdc-096511c9db7d None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:11.525644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04c0ec-9d72-4709-8f68-c7014ab28bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:11.527832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04c0ec-9d72-4709-8f68-c7014ab28bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:11.528395 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04c0ec-9d72-4709-8f68-c7014ab28bed admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:11.528891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04c0ec-9d72-4709-8f68-c7014ab28bed admin admin] PATH_INFO: `/v3/users/cc830fb444c545d388cd171c220472e6` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:11.531169 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04c0ec-9d72-4709-8f68-c7014ab28bed admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc830fb444c545d388cd171c220472e6)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:11.534052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04c0ec-9d72-4709-8f68-c7014ab28bed admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:11.539624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72d23a-43ef-4c07-98df-7d75922fc82e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:11.540399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72d23a-43ef-4c07-98df-7d75922fc82e service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:11.540679 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72d23a-43ef-4c07-98df-7d75922fc82e service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:11.540950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72d23a-43ef-4c07-98df-7d75922fc82e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:11.548495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-af72d23a-43ef-4c07-98df-7d75922fc82e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:11.557546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:11.586109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72d23a-43ef-4c07-98df-7d75922fc82e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:11.587950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72d23a-43ef-4c07-98df-7d75922fc82e service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:11.616259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 889/1782] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:11.617161 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72d23a-43ef-4c07-98df-7d75922fc82e service nova] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:11.620947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-af72d23a-43ef-4c07-98df-7d75922fc82e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:11.643556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9b04c0ec-9d72-4709-8f68-c7014ab28bed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc830fb444c545d388cd171c220472e6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:11.666555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9b04c0ec-9d72-4709-8f68-c7014ab28bed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc830fb444c545d388cd171c220472e6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:11.667647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9b04c0ec-9d72-4709-8f68-c7014ab28bed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc830fb444c545d388cd171c220472e6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:11.668565 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9b04c0ec-9d72-4709-8f68-c7014ab28bed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc830fb444c545d388cd171c220472e6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:11.691148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9b04c0ec-9d72-4709-8f68-c7014ab28bed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc830fb444c545d388cd171c220472e6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:11.726288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 894/1783] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:11 2022] DELETE /identity/v3/users/cc830fb444c545d388cd171c220472e6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:11.731132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f34fe-1dcd-40cd-a0fe-d28fe369c7db None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:11.731715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f34fe-1dcd-40cd-a0fe-d28fe369c7db None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:11.731825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f34fe-1dcd-40cd-a0fe-d28fe369c7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:11.741560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72d23a-43ef-4c07-98df-7d75922fc82e service nova] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:11.779440 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-198f34fe-1dcd-40cd-a0fe-d28fe369c7db None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:11.782765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-198f34fe-1dcd-40cd-a0fe-d28fe369c7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:11.791386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:11.840504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf019f2-1cd0-4fc6-9f62-8dac77cecbaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:11.841466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf019f2-1cd0-4fc6-9f62-8dac77cecbaa service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:11.841466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf019f2-1cd0-4fc6-9f62-8dac77cecbaa service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:11.841752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf019f2-1cd0-4fc6-9f62-8dac77cecbaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:11.844586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-acf019f2-1cd0-4fc6-9f62-8dac77cecbaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:11.984475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 895/1784] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:12.005444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f34fe-1dcd-40cd-a0fe-d28fe369c7db None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:12.009113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-198f34fe-1dcd-40cd-a0fe-d28fe369c7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:12.037808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f34fe-1dcd-40cd-a0fe-d28fe369c7db None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:12.045227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf019f2-1cd0-4fc6-9f62-8dac77cecbaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:12.048451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf019f2-1cd0-4fc6-9f62-8dac77cecbaa service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:12.051784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3030aa7-8ae6-4bb8-8db2-e629c828271a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:12.053619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3030aa7-8ae6-4bb8-8db2-e629c828271a service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:12.053897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3030aa7-8ae6-4bb8-8db2-e629c828271a service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:12.054172 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3030aa7-8ae6-4bb8-8db2-e629c828271a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:12.068390 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3030aa7-8ae6-4bb8-8db2-e629c828271a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:12.070144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3030aa7-8ae6-4bb8-8db2-e629c828271a service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:12.072533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 890/1785] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:12.082808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 896/1786] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:13.065595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf019f2-1cd0-4fc6-9f62-8dac77cecbaa service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:13.068534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-acf019f2-1cd0-4fc6-9f62-8dac77cecbaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:13.096987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3030aa7-8ae6-4bb8-8db2-e629c828271a service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:13.098509 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b3030aa7-8ae6-4bb8-8db2-e629c828271a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:13.102288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf019f2-1cd0-4fc6-9f62-8dac77cecbaa service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:13.115254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:13.117114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:13.118027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:13.118950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] PATH_INFO: `/v3/projects/47c39520582149cda6c44ecf57efa9b8` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:13.121706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3030aa7-8ae6-4bb8-8db2-e629c828271a service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:13.121954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47c39520582149cda6c44ecf57efa9b8)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:13.127035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:13.142257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:13.142914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:13.143195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:13.143474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] PATH_INFO: `/v3/projects/f4ffa19da6de49398c88ba3c2b9d2b43` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:13.149358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4ffa19da6de49398c88ba3c2b9d2b43)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:13.151981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:13.232095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4ffa19da6de49398c88ba3c2b9d2b43', 'request_id': 'req-6f5d98b9-470a-44c6-8082-938ef167f838'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.239668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4ffa19da6de49398c88ba3c2b9d2b43', 'request_id': 'req-6f5d98b9-470a-44c6-8082-938ef167f838'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.253319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4ffa19da6de49398c88ba3c2b9d2b43', 'request_id': 'req-6f5d98b9-470a-44c6-8082-938ef167f838'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.257322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47c39520582149cda6c44ecf57efa9b8', 'request_id': 'req-62638fed-47da-43d9-a1a8-349275472752'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.267864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47c39520582149cda6c44ecf57efa9b8', 'request_id': 'req-62638fed-47da-43d9-a1a8-349275472752'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.270889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] The token cache is being invalidate because project f4ffa19da6de49398c88ba3c2b9d2b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:13.271370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.272395 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.275162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 897/1787] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:13 2022] DELETE /identity/v3/projects/f4ffa19da6de49398c88ba3c2b9d2b43 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:13.276473 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47c39520582149cda6c44ecf57efa9b8', 'request_id': 'req-62638fed-47da-43d9-a1a8-349275472752'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.280878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:13.285101 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:13.290943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:13.291764 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] The token cache is being invalidate because project 47c39520582149cda6c44ecf57efa9b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:13.292150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.292961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.294810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 891/1788] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:13 2022] DELETE /identity/v3/projects/47c39520582149cda6c44ecf57efa9b8 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:13.298894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:13.300306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:13.351446 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d98b9-470a-44c6-8082-938ef167f838 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:13.356700 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:13.357454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:13.357831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:13.358196 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] PATH_INFO: `/v3/users/42e4bdf302394f9a98fa31994c5bc3e4` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:13.359410 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=42e4bdf302394f9a98fa31994c5bc3e4)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:13.361451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:13.369527 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62638fed-47da-43d9-a1a8-349275472752 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:13.377401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:13.378392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:13.378731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:13.378948 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] PATH_INFO: `/v3/users/e2cda203984445ea9c1ed21d27ee5646` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:13.380611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2cda203984445ea9c1ed21d27ee5646)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:13.382916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:13.390709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:13.423896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42e4bdf302394f9a98fa31994c5bc3e4'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.440646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42e4bdf302394f9a98fa31994c5bc3e4'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.446736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42e4bdf302394f9a98fa31994c5bc3e4'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.446736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42e4bdf302394f9a98fa31994c5bc3e4'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.454355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2cda203984445ea9c1ed21d27ee5646'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.455458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42e4bdf302394f9a98fa31994c5bc3e4'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.461615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2cda203984445ea9c1ed21d27ee5646'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.462454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2cda203984445ea9c1ed21d27ee5646'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.469535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 898/1789] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:13 2022] DELETE /identity/v3/users/42e4bdf302394f9a98fa31994c5bc3e4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:13.477189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2cda203984445ea9c1ed21d27ee5646'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.486897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2cda203984445ea9c1ed21d27ee5646'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:13.493434 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 892/1790] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:13 2022] DELETE /identity/v3/users/e2cda203984445ea9c1ed21d27ee5646 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:13.803436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:13.804937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:13.817882 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:13.819709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:13.909026 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e5057-4b1b-4148-bd91-1e451a9d72c2 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:13.912855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926eec-de9d-40c9-9ef8-991ddfce2e1c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:13.914273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2c668-cab1-48b9-879a-adef2672e067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:13.915161 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2c668-cab1-48b9-879a-adef2672e067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:13.915559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2c668-cab1-48b9-879a-adef2672e067 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:13.915877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2c668-cab1-48b9-879a-adef2672e067 admin admin] PATH_INFO: `/v3/projects/86e042aa756245d99acf36c161354aed` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:13.920262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2c668-cab1-48b9-879a-adef2672e067 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86e042aa756245d99acf36c161354aed)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:13.922384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2c668-cab1-48b9-879a-adef2672e067 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:13.923845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255136db-7827-457f-9291-209956bde42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:13.924790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255136db-7827-457f-9291-209956bde42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:13.925270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255136db-7827-457f-9291-209956bde42c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:13.925735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255136db-7827-457f-9291-209956bde42c admin admin] PATH_INFO: `/v3/projects/d2eba23fd55f45949ccf286476bbb068` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:13.927296 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255136db-7827-457f-9291-209956bde42c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2eba23fd55f45949ccf286476bbb068)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:13.929459 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255136db-7827-457f-9291-209956bde42c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:13.975481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:13.997443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ecf2c668-cab1-48b9-879a-adef2672e067 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86e042aa756245d99acf36c161354aed', 'request_id': 'req-ecf2c668-cab1-48b9-879a-adef2672e067'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:14.004823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ecf2c668-cab1-48b9-879a-adef2672e067 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86e042aa756245d99acf36c161354aed', 'request_id': 'req-ecf2c668-cab1-48b9-879a-adef2672e067'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:14.011411 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-255136db-7827-457f-9291-209956bde42c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2eba23fd55f45949ccf286476bbb068', 'request_id': 'req-255136db-7827-457f-9291-209956bde42c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:14.014609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ecf2c668-cab1-48b9-879a-adef2672e067 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86e042aa756245d99acf36c161354aed', 'request_id': 'req-ecf2c668-cab1-48b9-879a-adef2672e067'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:14.017992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-255136db-7827-457f-9291-209956bde42c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2eba23fd55f45949ccf286476bbb068', 'request_id': 'req-255136db-7827-457f-9291-209956bde42c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:14.025542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-255136db-7827-457f-9291-209956bde42c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2eba23fd55f45949ccf286476bbb068', 'request_id': 'req-255136db-7827-457f-9291-209956bde42c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:14.031124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ecf2c668-cab1-48b9-879a-adef2672e067 admin admin] The token cache is being invalidate because project 86e042aa756245d99acf36c161354aed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:14.031718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ecf2c668-cab1-48b9-879a-adef2672e067 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:14.032569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ecf2c668-cab1-48b9-879a-adef2672e067 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:14.034501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 893/1791] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:13 2022] DELETE /identity/v3/projects/86e042aa756245d99acf36c161354aed => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:14.039504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f82e3-b64e-4993-b82d-abf2cd565caa None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.039845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f82e3-b64e-4993-b82d-abf2cd565caa None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.040118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f82e3-b64e-4993-b82d-abf2cd565caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.041473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-255136db-7827-457f-9291-209956bde42c admin admin] The token cache is being invalidate because project d2eba23fd55f45949ccf286476bbb068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:14.042519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-255136db-7827-457f-9291-209956bde42c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:14.043474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-255136db-7827-457f-9291-209956bde42c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:14.045656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 899/1792] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:13 2022] DELETE /identity/v3/projects/d2eba23fd55f45949ccf286476bbb068 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:14.050429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e72d4d-b679-4a6a-896a-ac4fd8d3966d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.050637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e72d4d-b679-4a6a-896a-ac4fd8d3966d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.050916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e72d4d-b679-4a6a-896a-ac4fd8d3966d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.068480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-fa8f82e3-b64e-4993-b82d-abf2cd565caa None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:14.070437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-fa8f82e3-b64e-4993-b82d-abf2cd565caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:14.072815 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f5e72d4d-b679-4a6a-896a-ac4fd8d3966d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:14.074099 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f5e72d4d-b679-4a6a-896a-ac4fd8d3966d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:14.082801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:14.148839 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 894/1793] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:14.159750 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 900/1794] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:14.160311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de754-1d41-4c38-8a6b-8d2d0b942f17 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.160311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de754-1d41-4c38-8a6b-8d2d0b942f17 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.160311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de754-1d41-4c38-8a6b-8d2d0b942f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.163926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b04da-22a5-4f4c-a52c-06abe36bc2d6 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.164218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b04da-22a5-4f4c-a52c-06abe36bc2d6 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.164502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b04da-22a5-4f4c-a52c-06abe36bc2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.185390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-f90de754-1d41-4c38-8a6b-8d2d0b942f17 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:14.187010 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f90de754-1d41-4c38-8a6b-8d2d0b942f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:14.189632 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-3d4b04da-22a5-4f4c-a52c-06abe36bc2d6 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:14.190994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3d4b04da-22a5-4f4c-a52c-06abe36bc2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:14.199969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 901/1795] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:14.207334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b04da-22a5-4f4c-a52c-06abe36bc2d6 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.207475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 895/1796] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:14.208588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90de754-1d41-4c38-8a6b-8d2d0b942f17 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.215314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b04da-22a5-4f4c-a52c-06abe36bc2d6 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:14.229550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b90335-2faa-4974-9f79-1a8608e1b55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:14.232019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90de754-1d41-4c38-8a6b-8d2d0b942f17 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:14.232716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b90335-2faa-4974-9f79-1a8608e1b55f admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.233196 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b90335-2faa-4974-9f79-1a8608e1b55f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.233686 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b90335-2faa-4974-9f79-1a8608e1b55f admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.234674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b90335-2faa-4974-9f79-1a8608e1b55f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:14.235047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b90335-2faa-4974-9f79-1a8608e1b55f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:14.236791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b90335-2faa-4974-9f79-1a8608e1b55f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:14.251326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 902/1797] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:14.253371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae4ce-4a84-4182-b7be-4c1bd65dfa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:14.254370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae4ce-4a84-4182-b7be-4c1bd65dfa16 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.254797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae4ce-4a84-4182-b7be-4c1bd65dfa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.255253 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae4ce-4a84-4182-b7be-4c1bd65dfa16 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.256151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eae4ce-4a84-4182-b7be-4c1bd65dfa16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:14.256418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b90335-2faa-4974-9f79-1a8608e1b55f admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.256635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eae4ce-4a84-4182-b7be-4c1bd65dfa16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:14.260403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eae4ce-4a84-4182-b7be-4c1bd65dfa16 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:14.271335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b90335-2faa-4974-9f79-1a8608e1b55f admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:14.278287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547ffd4-b633-4f60-bb72-a0bf016519cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:14.279026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547ffd4-b633-4f60-bb72-a0bf016519cf admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.279153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547ffd4-b633-4f60-bb72-a0bf016519cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.279508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547ffd4-b633-4f60-bb72-a0bf016519cf admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.279833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 896/1798] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:14.280223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547ffd4-b633-4f60-bb72-a0bf016519cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:14.282259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547ffd4-b633-4f60-bb72-a0bf016519cf admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:14.284429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae4ce-4a84-4182-b7be-4c1bd65dfa16 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.297268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae4ce-4a84-4182-b7be-4c1bd65dfa16 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:14.303406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d8cd8-2fe0-4eb9-a0d4-5415f3565708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:14.304074 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d8cd8-2fe0-4eb9-a0d4-5415f3565708 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.304355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d8cd8-2fe0-4eb9-a0d4-5415f3565708 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.304630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d8cd8-2fe0-4eb9-a0d4-5415f3565708 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.305412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d8cd8-2fe0-4eb9-a0d4-5415f3565708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:14.307607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d8cd8-2fe0-4eb9-a0d4-5415f3565708 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:14.327040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 903/1799] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:14 2022] POST /identity/v3/projects => generated 361 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:14.330918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547ffd4-b633-4f60-bb72-a0bf016519cf admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.347417 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 897/1800] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:14 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:14.348692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d8cd8-2fe0-4eb9-a0d4-5415f3565708 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.430971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547ffd4-b633-4f60-bb72-a0bf016519cf admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:14.449683 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:14.494269 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d8cd8-2fe0-4eb9-a0d4-5415f3565708 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:14.509627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7f7d2-fba9-407b-8046-d17f361aee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:14.510265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7f7d2-fba9-407b-8046-d17f361aee10 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.511078 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7f7d2-fba9-407b-8046-d17f361aee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.511363 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7f7d2-fba9-407b-8046-d17f361aee10 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.512029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7f7d2-fba9-407b-8046-d17f361aee10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:14.514173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7f7d2-fba9-407b-8046-d17f361aee10 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:14.535734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b2662-0572-4120-a295-7393882c77de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:14.536815 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2662-0572-4120-a295-7393882c77de admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.537401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2662-0572-4120-a295-7393882c77de admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.538044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2662-0572-4120-a295-7393882c77de admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.539207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b2662-0572-4120-a295-7393882c77de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:14.541643 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b2662-0572-4120-a295-7393882c77de admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:14.559145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 898/1801] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:14 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:14.561736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7f7d2-fba9-407b-8046-d17f361aee10 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.578258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7f7d2-fba9-407b-8046-d17f361aee10 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:14.586289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8a637-36e7-4655-b721-0957e59d8a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:14.587260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8a637-36e7-4655-b721-0957e59d8a5e admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.587627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8a637-36e7-4655-b721-0957e59d8a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.587986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8a637-36e7-4655-b721-0957e59d8a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.589627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8a637-36e7-4655-b721-0957e59d8a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:14.591900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8a637-36e7-4655-b721-0957e59d8a5e admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:14.599182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 904/1802] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:14 2022] POST /identity/v3/users => generated 281 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:14.600881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b2662-0572-4120-a295-7393882c77de admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.610116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 899/1803] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:14.611126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8a637-36e7-4655-b721-0957e59d8a5e admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.618388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b2662-0572-4120-a295-7393882c77de admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:14.625687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8a637-36e7-4655-b721-0957e59d8a5e admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:14.627314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ceab7-409a-4924-a4be-a2089ed9a318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:14.628550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ceab7-409a-4924-a4be-a2089ed9a318 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.629362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ceab7-409a-4924-a4be-a2089ed9a318 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.630535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ceab7-409a-4924-a4be-a2089ed9a318 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.631562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ceab7-409a-4924-a4be-a2089ed9a318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:14.632066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d1c9d-c5b9-4ffb-890d-de6c53bd1459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:14.632875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d1c9d-c5b9-4ffb-890d-de6c53bd1459 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.633156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d1c9d-c5b9-4ffb-890d-de6c53bd1459 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.633433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d1c9d-c5b9-4ffb-890d-de6c53bd1459 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.634387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d1c9d-c5b9-4ffb-890d-de6c53bd1459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:14.635061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ceab7-409a-4924-a4be-a2089ed9a318 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:14.637592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d1c9d-c5b9-4ffb-890d-de6c53bd1459 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:14.656069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 905/1804] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:14.657149 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 900/1805] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:14.658234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d1c9d-c5b9-4ffb-890d-de6c53bd1459 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.658885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ceab7-409a-4924-a4be-a2089ed9a318 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.673249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d1c9d-c5b9-4ffb-890d-de6c53bd1459 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:14.675007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ceab7-409a-4924-a4be-a2089ed9a318 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:14.678794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111afcc4-e90b-4f5a-b7fb-04a75f1b42ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:14.679503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111afcc4-e90b-4f5a-b7fb-04a75f1b42ab admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.679811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111afcc4-e90b-4f5a-b7fb-04a75f1b42ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.680079 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111afcc4-e90b-4f5a-b7fb-04a75f1b42ab admin admin] PATH_INFO: `/v3/projects/3f61dce5f98941999b68009404134842/users/a3aaaba1ba344064994edee4955a0481/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.683902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cabca-ee8a-4422-ac0f-1bcfe7fb93f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:14.684948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cabca-ee8a-4422-ac0f-1bcfe7fb93f7 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.685500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cabca-ee8a-4422-ac0f-1bcfe7fb93f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.686080 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cabca-ee8a-4422-ac0f-1bcfe7fb93f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.687382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299cabca-ee8a-4422-ac0f-1bcfe7fb93f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:14.689520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299cabca-ee8a-4422-ac0f-1bcfe7fb93f7 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:14.702124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111afcc4-e90b-4f5a-b7fb-04a75f1b42ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f61dce5f98941999b68009404134842, user_id=a3aaaba1ba344064994edee4955a0481, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:14.704481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111afcc4-e90b-4f5a-b7fb-04a75f1b42ab admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:14.709892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 906/1806] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:14.711632 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cabca-ee8a-4422-ac0f-1bcfe7fb93f7 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.728453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cabca-ee8a-4422-ac0f-1bcfe7fb93f7 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:14.733272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 901/1807] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:14 2022] PUT /identity/v3/projects/3f61dce5f98941999b68009404134842/users/a3aaaba1ba344064994edee4955a0481/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:14.746634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa66ae-6726-48de-abe3-e364a59a2c9e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.747996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa66ae-6726-48de-abe3-e364a59a2c9e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.748523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa66ae-6726-48de-abe3-e364a59a2c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.828502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-46fa66ae-6726-48de-abe3-e364a59a2c9e None None] MFA Rules not processed for user `a3aaaba1ba344064994edee4955a0481`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:14.833728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-46fa66ae-6726-48de-abe3-e364a59a2c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:14.842972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:14.850785 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f4c42-ad87-4a4d-8598-8deca90aed3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:14.851785 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f4c42-ad87-4a4d-8598-8deca90aed3f admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.853271 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f4c42-ad87-4a4d-8598-8deca90aed3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.854426 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f4c42-ad87-4a4d-8598-8deca90aed3f admin admin] PATH_INFO: `/v3/projects/fb84e10ce5534a68b4a36b1bcb2eef0f/users/8b7cd11cbbb1415487953523d8aaf822/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.878285 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f4c42-ad87-4a4d-8598-8deca90aed3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb84e10ce5534a68b4a36b1bcb2eef0f, user_id=8b7cd11cbbb1415487953523d8aaf822, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:14.881457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f4c42-ad87-4a4d-8598-8deca90aed3f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:14.905564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 907/1808] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:14 2022] PUT /identity/v3/projects/fb84e10ce5534a68b4a36b1bcb2eef0f/users/8b7cd11cbbb1415487953523d8aaf822/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:14.918105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32aeb6-5945-4d32-87ac-a728aadec07e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:14.918681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32aeb6-5945-4d32-87ac-a728aadec07e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:14.919222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32aeb6-5945-4d32-87ac-a728aadec07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:14.980005 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 902/1809] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:14.980744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-7d32aeb6-5945-4d32-87ac-a728aadec07e None None] MFA Rules not processed for user `8b7cd11cbbb1415487953523d8aaf822`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:14.981859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7d32aeb6-5945-4d32-87ac-a728aadec07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:14.996614 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa66ae-6726-48de-abe3-e364a59a2c9e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:14.999422 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-46fa66ae-6726-48de-abe3-e364a59a2c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:15.084088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:15.100706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 908/1810] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:15.102121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32aeb6-5945-4d32-87ac-a728aadec07e None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:15.107283 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa66ae-6726-48de-abe3-e364a59a2c9e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:15.175712 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a344e88-9977-4d04-b2e7-d73c56dab227 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:15.176535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a344e88-9977-4d04-b2e7-d73c56dab227 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:15.177949 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a344e88-9977-4d04-b2e7-d73c56dab227 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:15.178113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a344e88-9977-4d04-b2e7-d73c56dab227 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:15.221048 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32aeb6-5945-4d32-87ac-a728aadec07e None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:15.237642 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa43edf-a37f-4391-bdcc-c86d8e88e265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:15.238598 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa43edf-a37f-4391-bdcc-c86d8e88e265 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:15.238598 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa43edf-a37f-4391-bdcc-c86d8e88e265 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:15.239355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa43edf-a37f-4391-bdcc-c86d8e88e265 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:15.240349 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa43edf-a37f-4391-bdcc-c86d8e88e265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:15.244123 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa43edf-a37f-4391-bdcc-c86d8e88e265 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:15.280106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a344e88-9977-4d04-b2e7-d73c56dab227 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:15.281873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a344e88-9977-4d04-b2e7-d73c56dab227 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:15.292683 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 903/1811] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:15.308800 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 909/1812] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:15 2022] POST /identity/v3/projects => generated 365 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:15.310812 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a344e88-9977-4d04-b2e7-d73c56dab227 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:15.326354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a344e88-9977-4d04-b2e7-d73c56dab227 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:15.343213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa43edf-a37f-4391-bdcc-c86d8e88e265 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:15.368743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:15.427061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f9528-c347-43d2-a8d8-2b456a1f097a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:15.427911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f9528-c347-43d2-a8d8-2b456a1f097a admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:15.428270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f9528-c347-43d2-a8d8-2b456a1f097a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:15.430093 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f9528-c347-43d2-a8d8-2b456a1f097a admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:15.430093 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f9528-c347-43d2-a8d8-2b456a1f097a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:15.433728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f9528-c347-43d2-a8d8-2b456a1f097a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:15.475143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 904/1813] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:15 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:15.479454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa43edf-a37f-4391-bdcc-c86d8e88e265 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:15.481230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f9528-c347-43d2-a8d8-2b456a1f097a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:15.490008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115b5da-9357-43d1-8e20-c1fc25f27d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:15.490805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115b5da-9357-43d1-8e20-c1fc25f27d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:15.491241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115b5da-9357-43d1-8e20-c1fc25f27d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:15.491528 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115b5da-9357-43d1-8e20-c1fc25f27d18 admin admin] PATH_INFO: `/v3/users/a3aaaba1ba344064994edee4955a0481` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:15.496996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f9528-c347-43d2-a8d8-2b456a1f097a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:15.509702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115b5da-9357-43d1-8e20-c1fc25f27d18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3aaaba1ba344064994edee4955a0481)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:15.512666 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8761-b2ca-4239-ad0f-57ed616bf81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:15.513880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115b5da-9357-43d1-8e20-c1fc25f27d18 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:15.520599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8761-b2ca-4239-ad0f-57ed616bf81e admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:15.520932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8761-b2ca-4239-ad0f-57ed616bf81e admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:15.521223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8761-b2ca-4239-ad0f-57ed616bf81e admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:15.521929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a8761-b2ca-4239-ad0f-57ed616bf81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:15.523618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a8761-b2ca-4239-ad0f-57ed616bf81e admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:15.558541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 905/1814] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:15.561198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8761-b2ca-4239-ad0f-57ed616bf81e admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:15.582693 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8761-b2ca-4239-ad0f-57ed616bf81e admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:15.601458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c31ce4-1363-4071-90e3-42f99f7ca2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:15.601931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c31ce4-1363-4071-90e3-42f99f7ca2ca admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:15.602214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c31ce4-1363-4071-90e3-42f99f7ca2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:15.602494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c31ce4-1363-4071-90e3-42f99f7ca2ca admin admin] PATH_INFO: `/v3/projects/c40c9d4272c34cb9b08e5e42af4db95a/users/233b973f398341c7b27d42beae4b21b1/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:15.645469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8115b5da-9357-43d1-8e20-c1fc25f27d18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3aaaba1ba344064994edee4955a0481'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:15.651569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c31ce4-1363-4071-90e3-42f99f7ca2ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40c9d4272c34cb9b08e5e42af4db95a, user_id=233b973f398341c7b27d42beae4b21b1, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:15.658651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c31ce4-1363-4071-90e3-42f99f7ca2ca admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:15.662559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8115b5da-9357-43d1-8e20-c1fc25f27d18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3aaaba1ba344064994edee4955a0481'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:15.663450 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8115b5da-9357-43d1-8e20-c1fc25f27d18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3aaaba1ba344064994edee4955a0481'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:15.664215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8115b5da-9357-43d1-8e20-c1fc25f27d18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3aaaba1ba344064994edee4955a0481'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:15.703732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 906/1815] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:15 2022] PUT /identity/v3/projects/c40c9d4272c34cb9b08e5e42af4db95a/users/233b973f398341c7b27d42beae4b21b1/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:15.705093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8115b5da-9357-43d1-8e20-c1fc25f27d18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3aaaba1ba344064994edee4955a0481'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:15.706146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c31ce4-1363-4071-90e3-42f99f7ca2ca admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:15.710715 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b4c31ce4-1363-4071-90e3-42f99f7ca2ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:15.746904 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 910/1816] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:15 2022] DELETE /identity/v3/users/a3aaaba1ba344064994edee4955a0481 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:15.750119 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26bd06-9ed3-48c6-8e36-a7a096e8cdbd None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:15.752280 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26bd06-9ed3-48c6-8e36-a7a096e8cdbd None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:15.752618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26bd06-9ed3-48c6-8e36-a7a096e8cdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:15.785927 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:15.826917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-8a26bd06-9ed3-48c6-8e36-a7a096e8cdbd None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:15.828778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8a26bd06-9ed3-48c6-8e36-a7a096e8cdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:15.930045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c31ce4-1363-4071-90e3-42f99f7ca2ca admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:16.011544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 911/1817] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:16.050022 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26bd06-9ed3-48c6-8e36-a7a096e8cdbd None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:16.052108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8a26bd06-9ed3-48c6-8e36-a7a096e8cdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:16.165070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333058d-b169-4a85-bee7-717d8f0f5cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:16.167662 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333058d-b169-4a85-bee7-717d8f0f5cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:16.168114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333058d-b169-4a85-bee7-717d8f0f5cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:16.168526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333058d-b169-4a85-bee7-717d8f0f5cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:16.170936 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333058d-b169-4a85-bee7-717d8f0f5cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:16.174052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333058d-b169-4a85-bee7-717d8f0f5cd2 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:16.197318 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 907/1818] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:16.199865 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26bd06-9ed3-48c6-8e36-a7a096e8cdbd None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:16.204771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333058d-b169-4a85-bee7-717d8f0f5cd2 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:16.210798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343ba03-896c-493d-a8ed-f2216d076b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:16.211434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343ba03-896c-493d-a8ed-f2216d076b0b service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:16.211732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343ba03-896c-493d-a8ed-f2216d076b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:16.212012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343ba03-896c-493d-a8ed-f2216d076b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:16.231161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343ba03-896c-493d-a8ed-f2216d076b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:16.232652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343ba03-896c-493d-a8ed-f2216d076b0b service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:16.233600 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333058d-b169-4a85-bee7-717d8f0f5cd2 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:16.244586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d65be6-2cde-4c69-8c41-7bc91bd38b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:16.245718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d65be6-2cde-4c69-8c41-7bc91bd38b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:16.246315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d65be6-2cde-4c69-8c41-7bc91bd38b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:16.246584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d65be6-2cde-4c69-8c41-7bc91bd38b56 admin admin] PATH_INFO: `/v3/projects/c40c9d4272c34cb9b08e5e42af4db95a/users/233b973f398341c7b27d42beae4b21b1/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:16.249827 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d65be6-2cde-4c69-8c41-7bc91bd38b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40c9d4272c34cb9b08e5e42af4db95a, user_id=233b973f398341c7b27d42beae4b21b1, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:16.253954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d65be6-2cde-4c69-8c41-7bc91bd38b56 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:16.260203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 912/1819] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:16.286394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 908/1820] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:16 2022] PUT /identity/v3/projects/c40c9d4272c34cb9b08e5e42af4db95a/users/233b973f398341c7b27d42beae4b21b1/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:16.296525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e11749-a2aa-4a2b-8acc-77a62b757b87 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:16.297110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e11749-a2aa-4a2b-8acc-77a62b757b87 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:16.297462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e11749-a2aa-4a2b-8acc-77a62b757b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:16.331264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-74e11749-a2aa-4a2b-8acc-77a62b757b87 None None] MFA Rules not processed for user `233b973f398341c7b27d42beae4b21b1`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:16.332889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-74e11749-a2aa-4a2b-8acc-77a62b757b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:16.449379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 913/1821] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:16.464630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d65be6-2cde-4c69-8c41-7bc91bd38b56 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:16.467112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-83d65be6-2cde-4c69-8c41-7bc91bd38b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:16.474480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:16.544505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d65be6-2cde-4c69-8c41-7bc91bd38b56 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:16.548015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56e180-aab3-4d85-8079-d3b90553e86d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:16.548703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56e180-aab3-4d85-8079-d3b90553e86d service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:16.549198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56e180-aab3-4d85-8079-d3b90553e86d service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:16.549954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56e180-aab3-4d85-8079-d3b90553e86d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:16.563269 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56e180-aab3-4d85-8079-d3b90553e86d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:16.564844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56e180-aab3-4d85-8079-d3b90553e86d service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:16.576926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 909/1822] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:16.665552 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e11749-a2aa-4a2b-8acc-77a62b757b87 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:16.675611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e11749-a2aa-4a2b-8acc-77a62b757b87 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:16.679721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa4baa-2573-4e90-9f68-55082c92e155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:16.680444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa4baa-2573-4e90-9f68-55082c92e155 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:16.680827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa4baa-2573-4e90-9f68-55082c92e155 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:16.681222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa4baa-2573-4e90-9f68-55082c92e155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:16.683132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b2fa4baa-2573-4e90-9f68-55082c92e155 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:16.733005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:16.768556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa4baa-2573-4e90-9f68-55082c92e155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:16.770007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa4baa-2573-4e90-9f68-55082c92e155 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:16.779813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 914/1823] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:16.796585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa402a7-cbce-4e48-a9d7-184581bde1dd None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:16.797083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa402a7-cbce-4e48-a9d7-184581bde1dd None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:16.797359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa402a7-cbce-4e48-a9d7-184581bde1dd None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:16.798833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 910/1824] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 14:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:52:17.004330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6509b-b057-4f25-a92e-d398c21c067a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:17.004892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6509b-b057-4f25-a92e-d398c21c067a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:17.005307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6509b-b057-4f25-a92e-d398c21c067a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:17.029078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-bfb6509b-b057-4f25-a92e-d398c21c067a None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:17.030466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bfb6509b-b057-4f25-a92e-d398c21c067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:17.118904 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 915/1825] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 14:52:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:17.358016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa402a7-cbce-4e48-a9d7-184581bde1dd None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:17.360162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-afa402a7-cbce-4e48-a9d7-184581bde1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:17.451965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6509b-b057-4f25-a92e-d398c21c067a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:17.453490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bfb6509b-b057-4f25-a92e-d398c21c067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:17.471772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6509b-b057-4f25-a92e-d398c21c067a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:17.478332 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b8863-c6a7-4c3d-81e0-b5ce93a83e06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:17.479008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b8863-c6a7-4c3d-81e0-b5ce93a83e06 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:17.479315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b8863-c6a7-4c3d-81e0-b5ce93a83e06 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:17.479590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b8863-c6a7-4c3d-81e0-b5ce93a83e06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:17.493208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b8863-c6a7-4c3d-81e0-b5ce93a83e06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:17.511720 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b8863-c6a7-4c3d-81e0-b5ce93a83e06 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:17.520569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 916/1826] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:17.540490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa402a7-cbce-4e48-a9d7-184581bde1dd None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:17.546785 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca01431-6e16-4960-90b1-4be179a41eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:17.547858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca01431-6e16-4960-90b1-4be179a41eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:17.548762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca01431-6e16-4960-90b1-4be179a41eba admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:17.549183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca01431-6e16-4960-90b1-4be179a41eba admin admin] PATH_INFO: `/v3/projects/3f61dce5f98941999b68009404134842` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:17.550459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca01431-6e16-4960-90b1-4be179a41eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f61dce5f98941999b68009404134842)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:17.552519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca01431-6e16-4960-90b1-4be179a41eba admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:17.611087 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-bca01431-6e16-4960-90b1-4be179a41eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f61dce5f98941999b68009404134842', 'request_id': 'req-bca01431-6e16-4960-90b1-4be179a41eba'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:17.617159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-bca01431-6e16-4960-90b1-4be179a41eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f61dce5f98941999b68009404134842', 'request_id': 'req-bca01431-6e16-4960-90b1-4be179a41eba'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:17.625805 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-bca01431-6e16-4960-90b1-4be179a41eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f61dce5f98941999b68009404134842', 'request_id': 'req-bca01431-6e16-4960-90b1-4be179a41eba'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:17.641256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-bca01431-6e16-4960-90b1-4be179a41eba admin admin] The token cache is being invalidate because project 3f61dce5f98941999b68009404134842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:17.641504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-bca01431-6e16-4960-90b1-4be179a41eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:17.642429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-bca01431-6e16-4960-90b1-4be179a41eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:17.645099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 911/1827] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:17 2022] DELETE /identity/v3/projects/3f61dce5f98941999b68009404134842 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:17.650338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec72dce-9db7-46fe-acbd-11827f16c7c9 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:17.650902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec72dce-9db7-46fe-acbd-11827f16c7c9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:17.651970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec72dce-9db7-46fe-acbd-11827f16c7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:17.678541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-cec72dce-9db7-46fe-acbd-11827f16c7c9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:17.680230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-cec72dce-9db7-46fe-acbd-11827f16c7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:17.802875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 917/1828] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:17.814129 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd680ac-5634-4824-8259-b351c68c18f5 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:17.814508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd680ac-5634-4824-8259-b351c68c18f5 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:17.814861 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd680ac-5634-4824-8259-b351c68c18f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:17.842942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-cdd680ac-5634-4824-8259-b351c68c18f5 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:17.844550 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-cdd680ac-5634-4824-8259-b351c68c18f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:17.855045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 912/1829] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:17.859817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec72dce-9db7-46fe-acbd-11827f16c7c9 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:17.880676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec72dce-9db7-46fe-acbd-11827f16c7c9 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:17.887805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0c83e-8825-4491-aa96-a012f73e34e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:17.888486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0c83e-8825-4491-aa96-a012f73e34e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:17.888776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0c83e-8825-4491-aa96-a012f73e34e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:17.889052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0c83e-8825-4491-aa96-a012f73e34e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:17.889715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0c83e-8825-4491-aa96-a012f73e34e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:17.890024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0c83e-8825-4491-aa96-a012f73e34e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:17.899927 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0c83e-8825-4491-aa96-a012f73e34e0 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:17.915787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 918/1830] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:17.918348 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd680ac-5634-4824-8259-b351c68c18f5 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:17.938639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd680ac-5634-4824-8259-b351c68c18f5 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:17.947406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4163-5c8b-434c-a1b0-50916a908edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:17.948671 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4163-5c8b-434c-a1b0-50916a908edb admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:17.948906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4163-5c8b-434c-a1b0-50916a908edb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:17.949176 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4163-5c8b-434c-a1b0-50916a908edb admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:17.949817 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4163-5c8b-434c-a1b0-50916a908edb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:17.952124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4163-5c8b-434c-a1b0-50916a908edb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:17.991937 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 913/1831] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:17 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:17.996735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0c83e-8825-4491-aa96-a012f73e34e0 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:18.009163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0c83e-8825-4491-aa96-a012f73e34e0 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:18.080525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:18.121240 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5264c8b-7cab-4c27-b5c8-266ea1f9c2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:18.121932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5264c8b-7cab-4c27-b5c8-266ea1f9c2ee admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:18.122134 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5264c8b-7cab-4c27-b5c8-266ea1f9c2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:18.122412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5264c8b-7cab-4c27-b5c8-266ea1f9c2ee admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:18.123048 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5264c8b-7cab-4c27-b5c8-266ea1f9c2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:18.125049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5264c8b-7cab-4c27-b5c8-266ea1f9c2ee admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:18.149940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 919/1832] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:17 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:18.154180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4163-5c8b-434c-a1b0-50916a908edb admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:18.165311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4163-5c8b-434c-a1b0-50916a908edb admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:18.170592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921bf91-8fb2-4f85-a854-247096f82356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:18.171207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921bf91-8fb2-4f85-a854-247096f82356 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:18.171489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921bf91-8fb2-4f85-a854-247096f82356 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:18.171801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921bf91-8fb2-4f85-a854-247096f82356 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:18.172455 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921bf91-8fb2-4f85-a854-247096f82356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:18.174274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921bf91-8fb2-4f85-a854-247096f82356 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:18.175938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:18.189434 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 914/1833] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:18.193718 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5264c8b-7cab-4c27-b5c8-266ea1f9c2ee admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:18.210030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5264c8b-7cab-4c27-b5c8-266ea1f9c2ee admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:18.217006 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca589284-e4fd-4b40-96c6-8a5bc38c22d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:18.217808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca589284-e4fd-4b40-96c6-8a5bc38c22d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:18.218283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca589284-e4fd-4b40-96c6-8a5bc38c22d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:18.218669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca589284-e4fd-4b40-96c6-8a5bc38c22d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:18.219650 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca589284-e4fd-4b40-96c6-8a5bc38c22d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:18.221477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca589284-e4fd-4b40-96c6-8a5bc38c22d6 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:18.235715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 920/1834] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:18.237856 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921bf91-8fb2-4f85-a854-247096f82356 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:18.248757 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921bf91-8fb2-4f85-a854-247096f82356 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:18.253921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5991f49-8e52-4189-ba82-55f3f985354e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:18.254637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5991f49-8e52-4189-ba82-55f3f985354e admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:18.254912 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5991f49-8e52-4189-ba82-55f3f985354e admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:18.255181 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5991f49-8e52-4189-ba82-55f3f985354e admin admin] PATH_INFO: `/v3/projects/76284b3bb22849e4a47df986926bd86a/users/abeb2acbe9e14adabb1d761f5dc89f1f/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:18.272316 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5991f49-8e52-4189-ba82-55f3f985354e admin admin] RBAC: Authorizing `identity:create_grant(project_id=76284b3bb22849e4a47df986926bd86a, user_id=abeb2acbe9e14adabb1d761f5dc89f1f, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:18.276378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5991f49-8e52-4189-ba82-55f3f985354e admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:18.312798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1d298-ebe2-4380-a37f-1f2e1fbd0e1e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:18.313066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1d298-ebe2-4380-a37f-1f2e1fbd0e1e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:18.313347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1d298-ebe2-4380-a37f-1f2e1fbd0e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:18.316106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 915/1835] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:18 2022] PUT /identity/v3/projects/76284b3bb22849e4a47df986926bd86a/users/abeb2acbe9e14adabb1d761f5dc89f1f/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:18.347402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-e3c1d298-ebe2-4380-a37f-1f2e1fbd0e1e None None] MFA Rules not processed for user `abeb2acbe9e14adabb1d761f5dc89f1f`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:18.349272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e3c1d298-ebe2-4380-a37f-1f2e1fbd0e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:18.487057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 921/1836] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:18.571630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5991f49-8e52-4189-ba82-55f3f985354e admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:18.574863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c5991f49-8e52-4189-ba82-55f3f985354e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:18.684404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5991f49-8e52-4189-ba82-55f3f985354e admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:18.691452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0906496-23a2-42ab-93bb-d51e3d927f9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:18.692877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0906496-23a2-42ab-93bb-d51e3d927f9f service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:18.693299 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0906496-23a2-42ab-93bb-d51e3d927f9f service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:18.693707 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0906496-23a2-42ab-93bb-d51e3d927f9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:18.704634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0906496-23a2-42ab-93bb-d51e3d927f9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:18.706084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0906496-23a2-42ab-93bb-d51e3d927f9f service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:18.717138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 916/1837] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:23.348682 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1d298-ebe2-4380-a37f-1f2e1fbd0e1e None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:23.350121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e3c1d298-ebe2-4380-a37f-1f2e1fbd0e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:23.449204 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1d298-ebe2-4380-a37f-1f2e1fbd0e1e None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:23.454298 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:23.455070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:23.455353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:23.455703 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] PATH_INFO: `/v3/users/b6b06275a0c94b8187fb1485154f57b8` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:23.456953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6b06275a0c94b8187fb1485154f57b8)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:23.459022 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:23.514593 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:23.517124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6b06275a0c94b8187fb1485154f57b8'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:23.527993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6b06275a0c94b8187fb1485154f57b8'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:23.528978 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6b06275a0c94b8187fb1485154f57b8'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:23.529728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6b06275a0c94b8187fb1485154f57b8'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:23.543791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6b06275a0c94b8187fb1485154f57b8'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:23.564124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 922/1838] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:23 2022] DELETE /identity/v3/users/b6b06275a0c94b8187fb1485154f57b8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:23.569105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efd0ad-d34d-4518-b124-529c67525091 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:23.569626 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efd0ad-d34d-4518-b124-529c67525091 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:23.569759 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efd0ad-d34d-4518-b124-529c67525091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:23.590207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-a3efd0ad-d34d-4518-b124-529c67525091 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:23.591423 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a3efd0ad-d34d-4518-b124-529c67525091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:23.648879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 917/1839] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:23.659292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:23.660791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:23.720093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bebf28-8cd3-44cf-829b-787b0547dfbb admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:23.723848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e6315-c06d-4ff2-81bf-b6b39a03dc17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:23.724468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e6315-c06d-4ff2-81bf-b6b39a03dc17 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:23.724749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e6315-c06d-4ff2-81bf-b6b39a03dc17 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:23.725021 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e6315-c06d-4ff2-81bf-b6b39a03dc17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:23.734563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e6315-c06d-4ff2-81bf-b6b39a03dc17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:23.736045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e6315-c06d-4ff2-81bf-b6b39a03dc17 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:23.746943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 923/1840] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:24.093881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efd0ad-d34d-4518-b124-529c67525091 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:24.095294 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a3efd0ad-d34d-4518-b124-529c67525091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:24.107570 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efd0ad-d34d-4518-b124-529c67525091 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:24.112348 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:24.112976 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:24.113254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:24.113530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] PATH_INFO: `/v3/projects/d21ddf0d090342588bedf10f6f1bd142` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:24.114699 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d21ddf0d090342588bedf10f6f1bd142)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:24.116628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:24.179517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd21ddf0d090342588bedf10f6f1bd142', 'request_id': 'req-12f0aee7-0a77-425b-9dee-bbeea06ae162'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.187962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd21ddf0d090342588bedf10f6f1bd142', 'request_id': 'req-12f0aee7-0a77-425b-9dee-bbeea06ae162'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.197968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd21ddf0d090342588bedf10f6f1bd142', 'request_id': 'req-12f0aee7-0a77-425b-9dee-bbeea06ae162'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.206701 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:24.212591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] The token cache is being invalidate because project d21ddf0d090342588bedf10f6f1bd142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:24.213166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.213962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.216732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 918/1841] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:24 2022] DELETE /identity/v3/projects/d21ddf0d090342588bedf10f6f1bd142 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:24.220715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e6315-c06d-4ff2-81bf-b6b39a03dc17 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:24.231416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e6315-c06d-4ff2-81bf-b6b39a03dc17 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:24.294663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433c9af-aadb-4fc0-bed1-97ed71b0acb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:24.295300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433c9af-aadb-4fc0-bed1-97ed71b0acb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:24.295573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433c9af-aadb-4fc0-bed1-97ed71b0acb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:24.295867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433c9af-aadb-4fc0-bed1-97ed71b0acb9 admin admin] PATH_INFO: `/v3/users/c9a46a7ab9554eb284fb8d8f9d3da202` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:24.296984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433c9af-aadb-4fc0-bed1-97ed71b0acb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a46a7ab9554eb284fb8d8f9d3da202)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:24.298898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433c9af-aadb-4fc0-bed1-97ed71b0acb9 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:24.353189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d433c9af-aadb-4fc0-bed1-97ed71b0acb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a46a7ab9554eb284fb8d8f9d3da202'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.362725 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d433c9af-aadb-4fc0-bed1-97ed71b0acb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a46a7ab9554eb284fb8d8f9d3da202'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.363492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d433c9af-aadb-4fc0-bed1-97ed71b0acb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a46a7ab9554eb284fb8d8f9d3da202'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.364143 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d433c9af-aadb-4fc0-bed1-97ed71b0acb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a46a7ab9554eb284fb8d8f9d3da202'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.373551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:24.379050 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d433c9af-aadb-4fc0-bed1-97ed71b0acb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a46a7ab9554eb284fb8d8f9d3da202'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.395060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 924/1842] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:24 2022] DELETE /identity/v3/users/c9a46a7ab9554eb284fb8d8f9d3da202 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:24.793934 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:24.795420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:24.808580 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0aee7-0a77-425b-9dee-bbeea06ae162 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:24.866913 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fca191-acb6-41f3-841f-509701a2804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:24.867529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fca191-acb6-41f3-841f-509701a2804d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:24.867844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fca191-acb6-41f3-841f-509701a2804d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:24.868119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fca191-acb6-41f3-841f-509701a2804d admin admin] PATH_INFO: `/v3/projects/bdcdc1361778439da3e1cec7964483fa` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:24.869241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fca191-acb6-41f3-841f-509701a2804d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdcdc1361778439da3e1cec7964483fa)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:24.871156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fca191-acb6-41f3-841f-509701a2804d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:24.920946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-30fca191-acb6-41f3-841f-509701a2804d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdcdc1361778439da3e1cec7964483fa', 'request_id': 'req-30fca191-acb6-41f3-841f-509701a2804d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.926559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-30fca191-acb6-41f3-841f-509701a2804d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdcdc1361778439da3e1cec7964483fa', 'request_id': 'req-30fca191-acb6-41f3-841f-509701a2804d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.934305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-30fca191-acb6-41f3-841f-509701a2804d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdcdc1361778439da3e1cec7964483fa', 'request_id': 'req-30fca191-acb6-41f3-841f-509701a2804d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.945569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-30fca191-acb6-41f3-841f-509701a2804d admin admin] The token cache is being invalidate because project bdcdc1361778439da3e1cec7964483fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:24.945783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-30fca191-acb6-41f3-841f-509701a2804d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.946536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-30fca191-acb6-41f3-841f-509701a2804d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:24.948496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 919/1843] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:24 2022] DELETE /identity/v3/projects/bdcdc1361778439da3e1cec7964483fa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:24.953898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13733ee-2161-4191-95f2-ee8ca417f706 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:24.954191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13733ee-2161-4191-95f2-ee8ca417f706 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:24.954469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13733ee-2161-4191-95f2-ee8ca417f706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:24.975770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f13733ee-2161-4191-95f2-ee8ca417f706 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:24.976559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f13733ee-2161-4191-95f2-ee8ca417f706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:25.036863 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 925/1844] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:25.045808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2c7a7-f89e-4ede-b620-876d35380b5a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.046088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2c7a7-f89e-4ede-b620-876d35380b5a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.046359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2c7a7-f89e-4ede-b620-876d35380b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.066053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-12b2c7a7-f89e-4ede-b620-876d35380b5a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:25.067284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-12b2c7a7-f89e-4ede-b620-876d35380b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:25.075750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 920/1845] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:25.080020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13733ee-2161-4191-95f2-ee8ca417f706 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:25.090358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13733ee-2161-4191-95f2-ee8ca417f706 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:25.094923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71874859-53bb-435a-b984-f29ba27edfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:25.095523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71874859-53bb-435a-b984-f29ba27edfad admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.095837 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71874859-53bb-435a-b984-f29ba27edfad admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.096114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71874859-53bb-435a-b984-f29ba27edfad admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.096776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71874859-53bb-435a-b984-f29ba27edfad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:25.097061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71874859-53bb-435a-b984-f29ba27edfad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:25.098651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71874859-53bb-435a-b984-f29ba27edfad admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:25.110185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 926/1846] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:25.114587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2c7a7-f89e-4ede-b620-876d35380b5a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:25.124153 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2c7a7-f89e-4ede-b620-876d35380b5a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:25.128704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0d2f3-86a6-464f-aecc-475db4bdfdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:25.129294 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0d2f3-86a6-464f-aecc-475db4bdfdf2 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.129566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0d2f3-86a6-464f-aecc-475db4bdfdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.129832 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0d2f3-86a6-464f-aecc-475db4bdfdf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.130446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0d2f3-86a6-464f-aecc-475db4bdfdf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:25.132691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0d2f3-86a6-464f-aecc-475db4bdfdf2 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:25.162688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 921/1847] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:25 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:25.166703 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71874859-53bb-435a-b984-f29ba27edfad admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:25.178253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71874859-53bb-435a-b984-f29ba27edfad admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:25.239650 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b9270-cc6d-4d4c-a99b-cc1c36da6e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:25.240205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b9270-cc6d-4d4c-a99b-cc1c36da6e20 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.240482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b9270-cc6d-4d4c-a99b-cc1c36da6e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.240752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b9270-cc6d-4d4c-a99b-cc1c36da6e20 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.241383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b9270-cc6d-4d4c-a99b-cc1c36da6e20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:25.243347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b9270-cc6d-4d4c-a99b-cc1c36da6e20 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:25.271748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 927/1848] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:25 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:25.272903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0d2f3-86a6-464f-aecc-475db4bdfdf2 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:25.283104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0d2f3-86a6-464f-aecc-475db4bdfdf2 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:25.288116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d3687-9178-4c50-92ed-7753d4649407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:25.288820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d3687-9178-4c50-92ed-7753d4649407 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.289161 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d3687-9178-4c50-92ed-7753d4649407 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.289441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d3687-9178-4c50-92ed-7753d4649407 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.290120 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d3687-9178-4c50-92ed-7753d4649407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:25.291788 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d3687-9178-4c50-92ed-7753d4649407 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:25.305289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 922/1849] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:25.308971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b9270-cc6d-4d4c-a99b-cc1c36da6e20 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:25.324500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b9270-cc6d-4d4c-a99b-cc1c36da6e20 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:25.329408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e4ea9-6b3f-42cd-bca0-5a514cc3afab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:25.330023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e4ea9-6b3f-42cd-bca0-5a514cc3afab admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.330311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e4ea9-6b3f-42cd-bca0-5a514cc3afab admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.330576 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e4ea9-6b3f-42cd-bca0-5a514cc3afab admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.331220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e4ea9-6b3f-42cd-bca0-5a514cc3afab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:25.333006 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e4ea9-6b3f-42cd-bca0-5a514cc3afab admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:25.344283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 928/1850] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:25.348500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d3687-9178-4c50-92ed-7753d4649407 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:25.358483 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d3687-9178-4c50-92ed-7753d4649407 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:25.363014 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bd27e-ae68-4478-87fe-1d2d0260437b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:25.363659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bd27e-ae68-4478-87fe-1d2d0260437b admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.363967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bd27e-ae68-4478-87fe-1d2d0260437b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.364241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bd27e-ae68-4478-87fe-1d2d0260437b admin admin] PATH_INFO: `/v3/projects/b289e02a48ae4add981c7eef02d01bf7/users/87dcd26c8cd9459c8225562336df8c73/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.379091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bd27e-ae68-4478-87fe-1d2d0260437b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b289e02a48ae4add981c7eef02d01bf7, user_id=87dcd26c8cd9459c8225562336df8c73, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:25.381768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bd27e-ae68-4478-87fe-1d2d0260437b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:25.396102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 923/1851] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:25 2022] PUT /identity/v3/projects/b289e02a48ae4add981c7eef02d01bf7/users/87dcd26c8cd9459c8225562336df8c73/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:25.406510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c72682-6ada-4cec-b1f5-4980ba666263 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.407041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c72682-6ada-4cec-b1f5-4980ba666263 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.407133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c72682-6ada-4cec-b1f5-4980ba666263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.440655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-49c72682-6ada-4cec-b1f5-4980ba666263 None None] MFA Rules not processed for user `87dcd26c8cd9459c8225562336df8c73`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:25.441952 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-49c72682-6ada-4cec-b1f5-4980ba666263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:25.486426 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:25.514015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 929/1852] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:25.518591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bd27e-ae68-4478-87fe-1d2d0260437b admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:25.545469 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:25.583341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bd27e-ae68-4478-87fe-1d2d0260437b admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:25.588097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b03693-beda-4154-aa4c-fd65657ed2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:25.588699 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b03693-beda-4154-aa4c-fd65657ed2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.588977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b03693-beda-4154-aa4c-fd65657ed2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.589245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b03693-beda-4154-aa4c-fd65657ed2d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.589875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b03693-beda-4154-aa4c-fd65657ed2d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:25.591801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b03693-beda-4154-aa4c-fd65657ed2d7 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:25.639424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 924/1853] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:25 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:25.649569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c72682-6ada-4cec-b1f5-4980ba666263 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:25.662097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c72682-6ada-4cec-b1f5-4980ba666263 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:25.733017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a71d9-f63d-4cef-9f60-98ef0ca93344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:25.733940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a71d9-f63d-4cef-9f60-98ef0ca93344 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.734387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a71d9-f63d-4cef-9f60-98ef0ca93344 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.734832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a71d9-f63d-4cef-9f60-98ef0ca93344 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.735775 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a71d9-f63d-4cef-9f60-98ef0ca93344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:25.738434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a71d9-f63d-4cef-9f60-98ef0ca93344 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:25.770792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 930/1854] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:25 2022] POST /identity/v3/users => generated 296 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:25.774733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b03693-beda-4154-aa4c-fd65657ed2d7 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:25.785230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b03693-beda-4154-aa4c-fd65657ed2d7 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:25.790101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fbe02-df72-43b5-8fa3-2db1c8a1b74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:25.790711 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fbe02-df72-43b5-8fa3-2db1c8a1b74e admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.790990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fbe02-df72-43b5-8fa3-2db1c8a1b74e admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.791260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fbe02-df72-43b5-8fa3-2db1c8a1b74e admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.792510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fbe02-df72-43b5-8fa3-2db1c8a1b74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:25.794123 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fbe02-df72-43b5-8fa3-2db1c8a1b74e admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:25.811198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 925/1855] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:25.812613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a71d9-f63d-4cef-9f60-98ef0ca93344 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:25.823372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a71d9-f63d-4cef-9f60-98ef0ca93344 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:25.828677 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b147190-c558-4eab-bd28-7ef20b860a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:25.829443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b147190-c558-4eab-bd28-7ef20b860a3d admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.829793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b147190-c558-4eab-bd28-7ef20b860a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.830146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b147190-c558-4eab-bd28-7ef20b860a3d admin admin] PATH_INFO: `/v3/projects/2bb1df7c575a42f79ea1a40900677444/users/46d8ace2f04147089657ea70d66296ab/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.848999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b147190-c558-4eab-bd28-7ef20b860a3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb1df7c575a42f79ea1a40900677444, user_id=46d8ace2f04147089657ea70d66296ab, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:25.851919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b147190-c558-4eab-bd28-7ef20b860a3d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:25.874398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 931/1856] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:25 2022] PUT /identity/v3/projects/2bb1df7c575a42f79ea1a40900677444/users/46d8ace2f04147089657ea70d66296ab/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:25.878825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fbe02-df72-43b5-8fa3-2db1c8a1b74e admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:25.902706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fbe02-df72-43b5-8fa3-2db1c8a1b74e admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:25.984762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01123f-4e7b-4c83-a152-1cfd912b3df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:25.985448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01123f-4e7b-4c83-a152-1cfd912b3df3 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:25.985732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01123f-4e7b-4c83-a152-1cfd912b3df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:25.986057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01123f-4e7b-4c83-a152-1cfd912b3df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:25.986700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01123f-4e7b-4c83-a152-1cfd912b3df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:25.988718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01123f-4e7b-4c83-a152-1cfd912b3df3 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:26.003873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 926/1857] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:26.005806 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b147190-c558-4eab-bd28-7ef20b860a3d admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:26.016784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b147190-c558-4eab-bd28-7ef20b860a3d admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:26.022473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68896af-f469-451f-8442-b2892e9a8f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:26.023229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68896af-f469-451f-8442-b2892e9a8f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:26.023413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68896af-f469-451f-8442-b2892e9a8f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:26.023725 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68896af-f469-451f-8442-b2892e9a8f09 admin admin] PATH_INFO: `/v3/projects/2bb1df7c575a42f79ea1a40900677444/users/46d8ace2f04147089657ea70d66296ab/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:26.028467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68896af-f469-451f-8442-b2892e9a8f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb1df7c575a42f79ea1a40900677444, user_id=46d8ace2f04147089657ea70d66296ab, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:26.031134 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68896af-f469-451f-8442-b2892e9a8f09 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:26.057102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 932/1858] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:26 2022] PUT /identity/v3/projects/2bb1df7c575a42f79ea1a40900677444/users/46d8ace2f04147089657ea70d66296ab/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:26.059542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e82e6-eb91-454e-9aab-1f68eb1b89bf None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:26.059863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e82e6-eb91-454e-9aab-1f68eb1b89bf None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:26.060143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e82e6-eb91-454e-9aab-1f68eb1b89bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:26.094850 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4e9e82e6-eb91-454e-9aab-1f68eb1b89bf None None] MFA Rules not processed for user `46d8ace2f04147089657ea70d66296ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:26.099754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4e9e82e6-eb91-454e-9aab-1f68eb1b89bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:26.175264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 927/1859] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:26.188827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68896af-f469-451f-8442-b2892e9a8f09 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:26.190562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d68896af-f469-451f-8442-b2892e9a8f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:26.248009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68896af-f469-451f-8442-b2892e9a8f09 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:26.251720 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3faa8-0a94-4ab8-a445-6ef776bfe74d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:26.252326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3faa8-0a94-4ab8-a445-6ef776bfe74d service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:26.252604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3faa8-0a94-4ab8-a445-6ef776bfe74d service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:26.252930 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3faa8-0a94-4ab8-a445-6ef776bfe74d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:26.263748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3faa8-0a94-4ab8-a445-6ef776bfe74d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:26.265198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3faa8-0a94-4ab8-a445-6ef776bfe74d service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:26.274902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 933/1860] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:26.621485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e82e6-eb91-454e-9aab-1f68eb1b89bf None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:26.631461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e82e6-eb91-454e-9aab-1f68eb1b89bf None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:26.637045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f976f52-a39d-4796-857e-28b0b5d3e93d tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46d8ace2f04147089657ea70d66296ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bb1df7c575a42f79ea1a40900677444', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:26.637658 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f976f52-a39d-4796-857e-28b0b5d3e93d tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:26.637942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f976f52-a39d-4796-857e-28b0b5d3e93d tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:26.638225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f976f52-a39d-4796-857e-28b0b5d3e93d tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:26.638899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f976f52-a39d-4796-857e-28b0b5d3e93d tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:26.639212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f976f52-a39d-4796-857e-28b0b5d3e93d tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:26.640863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f976f52-a39d-4796-857e-28b0b5d3e93d tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:26.657169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 928/1861] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:26.658547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3faa8-0a94-4ab8-a445-6ef776bfe74d service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:26.673689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3faa8-0a94-4ab8-a445-6ef776bfe74d service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:26.680172 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab493ed-8bee-4658-8fc4-d08510e352e3 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46d8ace2f04147089657ea70d66296ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bb1df7c575a42f79ea1a40900677444', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:26.681598 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab493ed-8bee-4658-8fc4-d08510e352e3 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:26.682823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab493ed-8bee-4658-8fc4-d08510e352e3 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:26.683117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab493ed-8bee-4658-8fc4-d08510e352e3 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:26.683951 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab493ed-8bee-4658-8fc4-d08510e352e3 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:26.686393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab493ed-8bee-4658-8fc4-d08510e352e3 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:26.725193 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 934/1862] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:26 2022] POST /identity/v3/projects => generated 354 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:26.730190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f976f52-a39d-4796-857e-28b0b5d3e93d tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:26.741754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f976f52-a39d-4796-857e-28b0b5d3e93d tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:26.747105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:26.812864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a09c58-8a61-48f3-8892-42baf52f11e7 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46d8ace2f04147089657ea70d66296ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bb1df7c575a42f79ea1a40900677444', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:26.813582 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a09c58-8a61-48f3-8892-42baf52f11e7 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:26.813852 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a09c58-8a61-48f3-8892-42baf52f11e7 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:26.814130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a09c58-8a61-48f3-8892-42baf52f11e7 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:26.814823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a09c58-8a61-48f3-8892-42baf52f11e7 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:26.815140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a09c58-8a61-48f3-8892-42baf52f11e7 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:26.816840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a09c58-8a61-48f3-8892-42baf52f11e7 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:26.830136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 929/1863] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:52:26.876564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab493ed-8bee-4658-8fc4-d08510e352e3 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:26.887295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab493ed-8bee-4658-8fc4-d08510e352e3 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:26.892382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d470d-40dc-48f0-a67d-7ef4e172bf08 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46d8ace2f04147089657ea70d66296ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bb1df7c575a42f79ea1a40900677444', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:26.893138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d470d-40dc-48f0-a67d-7ef4e172bf08 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:26.893493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d470d-40dc-48f0-a67d-7ef4e172bf08 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:26.893771 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d470d-40dc-48f0-a67d-7ef4e172bf08 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] PATH_INFO: `/v3/projects/c051ef8e759d401b88ba386d4a732f9d` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:26.894970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d470d-40dc-48f0-a67d-7ef4e172bf08 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c051ef8e759d401b88ba386d4a732f9d)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:26.896944 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d470d-40dc-48f0-a67d-7ef4e172bf08 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:26.907069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:26.951722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-012d470d-40dc-48f0-a67d-7ef4e172bf08 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c051ef8e759d401b88ba386d4a732f9d', 'request_id': 'req-012d470d-40dc-48f0-a67d-7ef4e172bf08'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:26.958248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-012d470d-40dc-48f0-a67d-7ef4e172bf08 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c051ef8e759d401b88ba386d4a732f9d', 'request_id': 'req-012d470d-40dc-48f0-a67d-7ef4e172bf08'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:26.967314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-012d470d-40dc-48f0-a67d-7ef4e172bf08 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c051ef8e759d401b88ba386d4a732f9d', 'request_id': 'req-012d470d-40dc-48f0-a67d-7ef4e172bf08'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:26.984791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-012d470d-40dc-48f0-a67d-7ef4e172bf08 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] The token cache is being invalidate because project c051ef8e759d401b88ba386d4a732f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:26.985261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-012d470d-40dc-48f0-a67d-7ef4e172bf08 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:26.985996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-012d470d-40dc-48f0-a67d-7ef4e172bf08 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:26.987709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 935/1864] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:26 2022] DELETE /identity/v3/projects/c051ef8e759d401b88ba386d4a732f9d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:27.215334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a09c58-8a61-48f3-8892-42baf52f11e7 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:27.217158 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b8a09c58-8a61-48f3-8892-42baf52f11e7 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:27.284346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a09c58-8a61-48f3-8892-42baf52f11e7 tempest-VolumeQuotaClassesTest-474519866 tempest-VolumeQuotaClassesTest-474519866-project-admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:27.341136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:27.341681 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:27.341960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:27.342235 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] PATH_INFO: `/v3/users/87dcd26c8cd9459c8225562336df8c73` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:27.343355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=87dcd26c8cd9459c8225562336df8c73)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:27.345311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:27.390007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87dcd26c8cd9459c8225562336df8c73'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:27.392064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:27.397968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87dcd26c8cd9459c8225562336df8c73'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:27.398977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87dcd26c8cd9459c8225562336df8c73'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:27.411546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87dcd26c8cd9459c8225562336df8c73'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:27.418977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87dcd26c8cd9459c8225562336df8c73'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:27.424838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 930/1865] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:27 2022] DELETE /identity/v3/users/87dcd26c8cd9459c8225562336df8c73 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:27.429367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdca90-be9e-4c98-aa65-4b7629e7e539 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:27.429655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdca90-be9e-4c98-aa65-4b7629e7e539 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:27.429933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdca90-be9e-4c98-aa65-4b7629e7e539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:27.450362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-dafdca90-be9e-4c98-aa65-4b7629e7e539 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:27.451580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-dafdca90-be9e-4c98-aa65-4b7629e7e539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:27.510107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 936/1866] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:27.520496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:27.522256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:27.581158 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ef53a-c91c-4be9-afcc-cfe8c484fccf admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:27.584851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094b8a0-ea79-476c-9d73-8005d86ca28e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:27.585462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094b8a0-ea79-476c-9d73-8005d86ca28e service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:27.585746 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094b8a0-ea79-476c-9d73-8005d86ca28e service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:27.586023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094b8a0-ea79-476c-9d73-8005d86ca28e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:27.595915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094b8a0-ea79-476c-9d73-8005d86ca28e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:27.597289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094b8a0-ea79-476c-9d73-8005d86ca28e service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:27.608461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 931/1867] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:27.929548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdca90-be9e-4c98-aa65-4b7629e7e539 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:27.931571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-dafdca90-be9e-4c98-aa65-4b7629e7e539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:27.944823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdca90-be9e-4c98-aa65-4b7629e7e539 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:27.950304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:27.950944 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:27.951224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:27.951499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] PATH_INFO: `/v3/projects/b289e02a48ae4add981c7eef02d01bf7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:27.952674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] RBAC: Authorizing `identity:delete_project(project_id=b289e02a48ae4add981c7eef02d01bf7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:27.954788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:28.008893 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b289e02a48ae4add981c7eef02d01bf7', 'request_id': 'req-4a254e17-4817-4542-a6d1-9441452d3cff'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.014393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b289e02a48ae4add981c7eef02d01bf7', 'request_id': 'req-4a254e17-4817-4542-a6d1-9441452d3cff'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.022211 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b289e02a48ae4add981c7eef02d01bf7', 'request_id': 'req-4a254e17-4817-4542-a6d1-9441452d3cff'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.033560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] The token cache is being invalidate because project b289e02a48ae4add981c7eef02d01bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:28.033847 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.034491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.036019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 937/1868] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:27 2022] DELETE /identity/v3/projects/b289e02a48ae4add981c7eef02d01bf7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:28.040101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094b8a0-ea79-476c-9d73-8005d86ca28e service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:28.050488 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094b8a0-ea79-476c-9d73-8005d86ca28e service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:28.111963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa7929-c862-459a-8a1a-c78490cbe1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:28.112674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa7929-c862-459a-8a1a-c78490cbe1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:28.112955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa7929-c862-459a-8a1a-c78490cbe1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:28.113248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa7929-c862-459a-8a1a-c78490cbe1a2 admin admin] PATH_INFO: `/v3/users/46d8ace2f04147089657ea70d66296ab` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:28.114436 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa7929-c862-459a-8a1a-c78490cbe1a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46d8ace2f04147089657ea70d66296ab)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:28.116410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa7929-c862-459a-8a1a-c78490cbe1a2 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:28.165844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-06aa7929-c862-459a-8a1a-c78490cbe1a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46d8ace2f04147089657ea70d66296ab'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.173570 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-06aa7929-c862-459a-8a1a-c78490cbe1a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46d8ace2f04147089657ea70d66296ab'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.174537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-06aa7929-c862-459a-8a1a-c78490cbe1a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46d8ace2f04147089657ea70d66296ab'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.186650 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:28.188004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-06aa7929-c862-459a-8a1a-c78490cbe1a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46d8ace2f04147089657ea70d66296ab'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.198200 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-06aa7929-c862-459a-8a1a-c78490cbe1a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46d8ace2f04147089657ea70d66296ab'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.205518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 932/1869] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:28 2022] DELETE /identity/v3/users/46d8ace2f04147089657ea70d66296ab => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:28.571865 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:28.574219 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:28.592856 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a254e17-4817-4542-a6d1-9441452d3cff admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:28.615848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:28.660711 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:28.661267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:28.661548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:28.661831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac admin admin] PATH_INFO: `/v3/projects/2bb1df7c575a42f79ea1a40900677444` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:28.663017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb1df7c575a42f79ea1a40900677444)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:28.665111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:28.715206 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb1df7c575a42f79ea1a40900677444', 'request_id': 'req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.720674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb1df7c575a42f79ea1a40900677444', 'request_id': 'req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.728906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb1df7c575a42f79ea1a40900677444', 'request_id': 'req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.742538 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac admin admin] The token cache is being invalidate because project 2bb1df7c575a42f79ea1a40900677444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:28.742854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.743524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5dfaa5a6-5db2-4e4a-bfc0-6c90250334ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:28.745141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 938/1870] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:28 2022] DELETE /identity/v3/projects/2bb1df7c575a42f79ea1a40900677444 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:28.750087 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0a305-190f-43e9-ab02-c29947e09256 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:28.750381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0a305-190f-43e9-ab02-c29947e09256 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:28.750663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0a305-190f-43e9-ab02-c29947e09256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:28.778114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-55d0a305-190f-43e9-ab02-c29947e09256 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:28.779352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-55d0a305-190f-43e9-ab02-c29947e09256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:28.870057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 933/1871] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:28.871784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5a642-1d05-41c8-a3cb-bf5ff519e6c5 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:28.872070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5a642-1d05-41c8-a3cb-bf5ff519e6c5 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:28.872346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5a642-1d05-41c8-a3cb-bf5ff519e6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:28.904996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-6aa5a642-1d05-41c8-a3cb-bf5ff519e6c5 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:28.920383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6aa5a642-1d05-41c8-a3cb-bf5ff519e6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:28.929753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 939/1872] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:28.934731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0a305-190f-43e9-ab02-c29947e09256 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:28.944674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0a305-190f-43e9-ab02-c29947e09256 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:28.949403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e700187-d391-44f5-8218-0eb4b15a262d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:28.950012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e700187-d391-44f5-8218-0eb4b15a262d admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:28.950299 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e700187-d391-44f5-8218-0eb4b15a262d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:28.950573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e700187-d391-44f5-8218-0eb4b15a262d admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:28.951255 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e700187-d391-44f5-8218-0eb4b15a262d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:28.951566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e700187-d391-44f5-8218-0eb4b15a262d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:28.953216 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e700187-d391-44f5-8218-0eb4b15a262d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:28.968669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 934/1873] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:52:28.973141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5a642-1d05-41c8-a3cb-bf5ff519e6c5 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:28.987106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5a642-1d05-41c8-a3cb-bf5ff519e6c5 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:28.993304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987c496-525a-4f8e-8623-684b2a9b5f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:28.994129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987c496-525a-4f8e-8623-684b2a9b5f79 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:28.994598 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987c496-525a-4f8e-8623-684b2a9b5f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:28.995297 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987c496-525a-4f8e-8623-684b2a9b5f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:28.996620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987c496-525a-4f8e-8623-684b2a9b5f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:29.000884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987c496-525a-4f8e-8623-684b2a9b5f79 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:29.050152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 940/1874] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:28 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:29.055910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e700187-d391-44f5-8218-0eb4b15a262d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:29.067048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e700187-d391-44f5-8218-0eb4b15a262d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:29.126185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67d14a-f708-4629-8c73-559349661f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:29.126807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67d14a-f708-4629-8c73-559349661f60 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:29.127083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67d14a-f708-4629-8c73-559349661f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:29.127351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67d14a-f708-4629-8c73-559349661f60 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:29.128028 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67d14a-f708-4629-8c73-559349661f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:29.129950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67d14a-f708-4629-8c73-559349661f60 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:29.150007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 935/1875] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:29 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:29.154727 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987c496-525a-4f8e-8623-684b2a9b5f79 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:29.164909 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987c496-525a-4f8e-8623-684b2a9b5f79 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:29.170266 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dc61b-2a12-4b8e-b6e6-52224f4800aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:29.170795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dc61b-2a12-4b8e-b6e6-52224f4800aa admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:29.171076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dc61b-2a12-4b8e-b6e6-52224f4800aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:29.171349 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dc61b-2a12-4b8e-b6e6-52224f4800aa admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:29.172017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2dc61b-2a12-4b8e-b6e6-52224f4800aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:29.173608 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2dc61b-2a12-4b8e-b6e6-52224f4800aa admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:29.184084 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 941/1876] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:29.188174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67d14a-f708-4629-8c73-559349661f60 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:29.198643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67d14a-f708-4629-8c73-559349661f60 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:29.203249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d52819-d30e-4f4d-afcf-6113c71fc9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:29.203886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d52819-d30e-4f4d-afcf-6113c71fc9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:29.204180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d52819-d30e-4f4d-afcf-6113c71fc9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:29.204430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d52819-d30e-4f4d-afcf-6113c71fc9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:29.205098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d52819-d30e-4f4d-afcf-6113c71fc9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:29.206694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d52819-d30e-4f4d-afcf-6113c71fc9f3 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:29.217921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 936/1877] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:29.222206 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dc61b-2a12-4b8e-b6e6-52224f4800aa admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:29.233079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dc61b-2a12-4b8e-b6e6-52224f4800aa admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:29.237837 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b8c10-f05a-463e-9189-d05d197dbc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:29.238524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b8c10-f05a-463e-9189-d05d197dbc84 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:29.238803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b8c10-f05a-463e-9189-d05d197dbc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:29.239077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b8c10-f05a-463e-9189-d05d197dbc84 admin admin] PATH_INFO: `/v3/projects/cda2735fb7ea4e25aae786534e933c2c/users/be429f537d0849d98e4d9b8f6ac1bc30/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:29.258377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b8c10-f05a-463e-9189-d05d197dbc84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda2735fb7ea4e25aae786534e933c2c, user_id=be429f537d0849d98e4d9b8f6ac1bc30, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:29.261161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b8c10-f05a-463e-9189-d05d197dbc84 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:29.277030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 942/1878] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:29 2022] PUT /identity/v3/projects/cda2735fb7ea4e25aae786534e933c2c/users/be429f537d0849d98e4d9b8f6ac1bc30/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:29.287458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfacee-c90e-4486-b3f6-456fdcbd89e8 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:29.287767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfacee-c90e-4486-b3f6-456fdcbd89e8 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:29.288047 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfacee-c90e-4486-b3f6-456fdcbd89e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:29.322031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8bbfacee-c90e-4486-b3f6-456fdcbd89e8 None None] MFA Rules not processed for user `be429f537d0849d98e4d9b8f6ac1bc30`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:29.323797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8bbfacee-c90e-4486-b3f6-456fdcbd89e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:29.379383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:29.423484 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 937/1879] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:29.424339 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b8c10-f05a-463e-9189-d05d197dbc84 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:29.499514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:29.540002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b8c10-f05a-463e-9189-d05d197dbc84 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:29.547005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90153008-be5b-4012-909f-dd24d4edbf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:29.547822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90153008-be5b-4012-909f-dd24d4edbf76 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:29.548122 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90153008-be5b-4012-909f-dd24d4edbf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:29.548561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90153008-be5b-4012-909f-dd24d4edbf76 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:29.549214 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90153008-be5b-4012-909f-dd24d4edbf76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:29.552027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90153008-be5b-4012-909f-dd24d4edbf76 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:29.590333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 943/1880] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:29 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:29.594629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfacee-c90e-4486-b3f6-456fdcbd89e8 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:29.610376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfacee-c90e-4486-b3f6-456fdcbd89e8 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:29.701199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1518a10-7671-4b9d-8fb8-11bf07c82579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:29.701855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1518a10-7671-4b9d-8fb8-11bf07c82579 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:29.702231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1518a10-7671-4b9d-8fb8-11bf07c82579 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:29.702623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1518a10-7671-4b9d-8fb8-11bf07c82579 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:29.703295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1518a10-7671-4b9d-8fb8-11bf07c82579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:29.705398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1518a10-7671-4b9d-8fb8-11bf07c82579 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:29.747869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 938/1881] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:29 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:29.751396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90153008-be5b-4012-909f-dd24d4edbf76 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:29.764981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90153008-be5b-4012-909f-dd24d4edbf76 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:29.773466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d3aa7-473d-4211-bbd4-73dc78acb277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:29.775398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d3aa7-473d-4211-bbd4-73dc78acb277 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:29.775724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d3aa7-473d-4211-bbd4-73dc78acb277 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:29.777200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d3aa7-473d-4211-bbd4-73dc78acb277 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:29.777967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d3aa7-473d-4211-bbd4-73dc78acb277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:29.781148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d3aa7-473d-4211-bbd4-73dc78acb277 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:29.812389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 944/1882] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:29.813301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1518a10-7671-4b9d-8fb8-11bf07c82579 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:29.826754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1518a10-7671-4b9d-8fb8-11bf07c82579 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:29.831797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503263d-7db3-45b2-8e5b-4c2a0519e7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:29.832433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503263d-7db3-45b2-8e5b-4c2a0519e7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:29.832713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503263d-7db3-45b2-8e5b-4c2a0519e7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:29.833086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503263d-7db3-45b2-8e5b-4c2a0519e7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:29.833966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503263d-7db3-45b2-8e5b-4c2a0519e7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:29.835848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503263d-7db3-45b2-8e5b-4c2a0519e7c1 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:29.852332 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d3aa7-473d-4211-bbd4-73dc78acb277 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:29.857656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 939/1883] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:29.863784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d3aa7-473d-4211-bbd4-73dc78acb277 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:29.868894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06305d42-c4cd-4c02-8d5c-9c41d3651432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:29.870232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06305d42-c4cd-4c02-8d5c-9c41d3651432 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:29.870571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06305d42-c4cd-4c02-8d5c-9c41d3651432 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:29.870907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06305d42-c4cd-4c02-8d5c-9c41d3651432 admin admin] PATH_INFO: `/v3/projects/de20524a38c44e57b2634d3f4d63e348/users/2bddfac2635045f0a7ea9fbcc6c5a9f2/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:29.890292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06305d42-c4cd-4c02-8d5c-9c41d3651432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de20524a38c44e57b2634d3f4d63e348, user_id=2bddfac2635045f0a7ea9fbcc6c5a9f2, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:29.893397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06305d42-c4cd-4c02-8d5c-9c41d3651432 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:29.911792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 945/1884] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:29 2022] PUT /identity/v3/projects/de20524a38c44e57b2634d3f4d63e348/users/2bddfac2635045f0a7ea9fbcc6c5a9f2/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:29.925293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb232d-5055-4ebd-afbc-e2cf9cb4bb7f None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:29.925544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb232d-5055-4ebd-afbc-e2cf9cb4bb7f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:29.925867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb232d-5055-4ebd-afbc-e2cf9cb4bb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:29.963407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3cdb232d-5055-4ebd-afbc-e2cf9cb4bb7f None None] MFA Rules not processed for user `2bddfac2635045f0a7ea9fbcc6c5a9f2`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:29.964878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3cdb232d-5055-4ebd-afbc-e2cf9cb4bb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:30.066043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 940/1885] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:30.071021 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06305d42-c4cd-4c02-8d5c-9c41d3651432 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:30.171025 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06305d42-c4cd-4c02-8d5c-9c41d3651432 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:30.176112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570fa14-0d87-4b77-943d-4345833c6be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:30.176824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570fa14-0d87-4b77-943d-4345833c6be5 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:30.177347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570fa14-0d87-4b77-943d-4345833c6be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:30.177748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570fa14-0d87-4b77-943d-4345833c6be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:30.178473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570fa14-0d87-4b77-943d-4345833c6be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:30.180669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570fa14-0d87-4b77-943d-4345833c6be5 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:30.232504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 946/1886] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:30 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:30.237474 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb232d-5055-4ebd-afbc-e2cf9cb4bb7f None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:30.269286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb232d-5055-4ebd-afbc-e2cf9cb4bb7f None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:30.381899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:30.397238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276c715-9020-453f-a994-e971099cd607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:30.398060 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276c715-9020-453f-a994-e971099cd607 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:30.398380 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276c715-9020-453f-a994-e971099cd607 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:30.398667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276c715-9020-453f-a994-e971099cd607 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:30.400142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276c715-9020-453f-a994-e971099cd607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:30.402413 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276c715-9020-453f-a994-e971099cd607 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:30.448874 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 941/1887] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:30 2022] POST /identity/v3/users => generated 299 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:30.453623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570fa14-0d87-4b77-943d-4345833c6be5 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:30.478114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570fa14-0d87-4b77-943d-4345833c6be5 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:30.491485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb544f7b-4d86-4393-b4c7-fba4d9fea45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:30.492160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb544f7b-4d86-4393-b4c7-fba4d9fea45c admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:30.492891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb544f7b-4d86-4393-b4c7-fba4d9fea45c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:30.493187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb544f7b-4d86-4393-b4c7-fba4d9fea45c admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:30.494305 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb544f7b-4d86-4393-b4c7-fba4d9fea45c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:30.495538 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb544f7b-4d86-4393-b4c7-fba4d9fea45c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:30.515757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 947/1888] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:30.522769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276c715-9020-453f-a994-e971099cd607 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:30.554268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276c715-9020-453f-a994-e971099cd607 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:30.569810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b62168-5a90-40de-9fce-47e71c2333d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:30.570772 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b62168-5a90-40de-9fce-47e71c2333d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:30.571741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b62168-5a90-40de-9fce-47e71c2333d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:30.572212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b62168-5a90-40de-9fce-47e71c2333d0 admin admin] PATH_INFO: `/v3/projects/279a7184a41847228c7741786b1ec087/users/eee90e29bad946829c5b85e530f0f5c0/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:30.618913 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b62168-5a90-40de-9fce-47e71c2333d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=279a7184a41847228c7741786b1ec087, user_id=eee90e29bad946829c5b85e530f0f5c0, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:30.622430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b62168-5a90-40de-9fce-47e71c2333d0 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:30.653546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 942/1889] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:30 2022] PUT /identity/v3/projects/279a7184a41847228c7741786b1ec087/users/eee90e29bad946829c5b85e530f0f5c0/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:30.656583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb544f7b-4d86-4393-b4c7-fba4d9fea45c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:30.669062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb544f7b-4d86-4393-b4c7-fba4d9fea45c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:30.761669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753aad1-1e78-4f57-84e6-c9bf0eff0aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:30.762457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753aad1-1e78-4f57-84e6-c9bf0eff0aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:30.762825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753aad1-1e78-4f57-84e6-c9bf0eff0aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:30.763195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753aad1-1e78-4f57-84e6-c9bf0eff0aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:30.766596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753aad1-1e78-4f57-84e6-c9bf0eff0aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:30.766596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753aad1-1e78-4f57-84e6-c9bf0eff0aa4 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:30.782954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 948/1890] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:30.793187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b62168-5a90-40de-9fce-47e71c2333d0 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:30.812682 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b62168-5a90-40de-9fce-47e71c2333d0 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:30.820361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f5e69-f2b4-4e01-ad24-d16968ec422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:30.821607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f5e69-f2b4-4e01-ad24-d16968ec422c admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:30.822084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f5e69-f2b4-4e01-ad24-d16968ec422c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:30.822547 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f5e69-f2b4-4e01-ad24-d16968ec422c admin admin] PATH_INFO: `/v3/projects/279a7184a41847228c7741786b1ec087/users/eee90e29bad946829c5b85e530f0f5c0/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:30.826566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f5e69-f2b4-4e01-ad24-d16968ec422c admin admin] RBAC: Authorizing `identity:create_grant(project_id=279a7184a41847228c7741786b1ec087, user_id=eee90e29bad946829c5b85e530f0f5c0, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:30.829767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f5e69-f2b4-4e01-ad24-d16968ec422c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:30.856646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 943/1891] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:30 2022] PUT /identity/v3/projects/279a7184a41847228c7741786b1ec087/users/eee90e29bad946829c5b85e530f0f5c0/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:30.869852 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bdda4-e9d9-48da-b63e-f61fc5fb3493 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:30.870515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bdda4-e9d9-48da-b63e-f61fc5fb3493 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:30.870980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bdda4-e9d9-48da-b63e-f61fc5fb3493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:30.893465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:30.925441 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-455bdda4-e9d9-48da-b63e-f61fc5fb3493 None None] MFA Rules not processed for user `eee90e29bad946829c5b85e530f0f5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:30.927812 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-455bdda4-e9d9-48da-b63e-f61fc5fb3493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:31.067884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 949/1892] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:31.142261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f5e69-f2b4-4e01-ad24-d16968ec422c admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:31.144130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-643f5e69-f2b4-4e01-ad24-d16968ec422c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:31.208800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f5e69-f2b4-4e01-ad24-d16968ec422c admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:31.212897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f0cc2-b08b-41e5-8210-2863a0039eaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:31.213688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0cc2-b08b-41e5-8210-2863a0039eaa service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:31.213971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0cc2-b08b-41e5-8210-2863a0039eaa service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:31.214245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0cc2-b08b-41e5-8210-2863a0039eaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:31.224223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f0cc2-b08b-41e5-8210-2863a0039eaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:31.225736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f0cc2-b08b-41e5-8210-2863a0039eaa service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:31.236517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 944/1893] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:31.313700 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bdda4-e9d9-48da-b63e-f61fc5fb3493 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:31.316000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-455bdda4-e9d9-48da-b63e-f61fc5fb3493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:31.420038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bdda4-e9d9-48da-b63e-f61fc5fb3493 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:31.425090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:31.426049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:31.426335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:31.426711 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] PATH_INFO: `/v3/users/1bdc7af040cb4449b556125327a0d9aa` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:31.428669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bdc7af040cb4449b556125327a0d9aa)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:31.430906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:31.507424 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bdc7af040cb4449b556125327a0d9aa'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:31.519459 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bdc7af040cb4449b556125327a0d9aa'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:31.520342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bdc7af040cb4449b556125327a0d9aa'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:31.521036 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bdc7af040cb4449b556125327a0d9aa'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:31.539907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bdc7af040cb4449b556125327a0d9aa'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:31.563052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 950/1894] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:31 2022] DELETE /identity/v3/users/1bdc7af040cb4449b556125327a0d9aa => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:31.569001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708744e-7052-4692-87b3-6b46a3dc1172 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:31.569558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708744e-7052-4692-87b3-6b46a3dc1172 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:31.572134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708744e-7052-4692-87b3-6b46a3dc1172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:31.599148 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-9708744e-7052-4692-87b3-6b46a3dc1172 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:31.600569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9708744e-7052-4692-87b3-6b46a3dc1172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:31.707465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 945/1895] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:31.723011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:31.725191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:31.769091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:31.878975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3ce73-27b7-49af-96a7-da51f3b0c259 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:31.883990 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb46c0c-bb5a-4c0e-8f50-6058538a3650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:31.884900 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb46c0c-bb5a-4c0e-8f50-6058538a3650 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:31.885437 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb46c0c-bb5a-4c0e-8f50-6058538a3650 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:31.886026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb46c0c-bb5a-4c0e-8f50-6058538a3650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:31.905199 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb46c0c-bb5a-4c0e-8f50-6058538a3650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:31.908264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb46c0c-bb5a-4c0e-8f50-6058538a3650 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:31.923236 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 951/1896] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:32.011542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708744e-7052-4692-87b3-6b46a3dc1172 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:32.013006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9708744e-7052-4692-87b3-6b46a3dc1172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:32.099267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:32.105568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708744e-7052-4692-87b3-6b46a3dc1172 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:32.118195 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889f7cf-3a45-4d61-be74-506ff8cbc455 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:32.120571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889f7cf-3a45-4d61-be74-506ff8cbc455 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:32.121063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889f7cf-3a45-4d61-be74-506ff8cbc455 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:32.121867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889f7cf-3a45-4d61-be74-506ff8cbc455 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:32.125453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f889f7cf-3a45-4d61-be74-506ff8cbc455 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:32.271482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889f7cf-3a45-4d61-be74-506ff8cbc455 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:32.273042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889f7cf-3a45-4d61-be74-506ff8cbc455 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:32.286264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 946/1897] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:32.336776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb46c0c-bb5a-4c0e-8f50-6058538a3650 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:32.339420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ecb46c0c-bb5a-4c0e-8f50-6058538a3650 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:32.374572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb46c0c-bb5a-4c0e-8f50-6058538a3650 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:32.395591 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:32.396733 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:32.397246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:32.397765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] PATH_INFO: `/v3/projects/cf6c45f9df6f4fdb809218830afbacbc` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:32.399327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf6c45f9df6f4fdb809218830afbacbc)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:32.406632 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:32.535080 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf6c45f9df6f4fdb809218830afbacbc', 'request_id': 'req-e1fa576c-2abd-499c-b21d-7ab2273de61c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:32.546919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf6c45f9df6f4fdb809218830afbacbc', 'request_id': 'req-e1fa576c-2abd-499c-b21d-7ab2273de61c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:32.561922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf6c45f9df6f4fdb809218830afbacbc', 'request_id': 'req-e1fa576c-2abd-499c-b21d-7ab2273de61c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:32.595910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] The token cache is being invalidate because project cf6c45f9df6f4fdb809218830afbacbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:32.596212 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:32.600433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:32.603325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 952/1898] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:32 2022] DELETE /identity/v3/projects/cf6c45f9df6f4fdb809218830afbacbc => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:32.607572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889f7cf-3a45-4d61-be74-506ff8cbc455 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:32.626243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889f7cf-3a45-4d61-be74-506ff8cbc455 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:32.730518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66e288-7c82-4022-a502-b1a182bfc047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:32.731171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66e288-7c82-4022-a502-b1a182bfc047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:32.731481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66e288-7c82-4022-a502-b1a182bfc047 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:32.731814 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66e288-7c82-4022-a502-b1a182bfc047 admin admin] PATH_INFO: `/v3/users/3c42c9aee1a249279e0856272d876548` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:32.733549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66e288-7c82-4022-a502-b1a182bfc047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c42c9aee1a249279e0856272d876548)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:32.735484 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66e288-7c82-4022-a502-b1a182bfc047 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:32.807719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3e66e288-7c82-4022-a502-b1a182bfc047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c42c9aee1a249279e0856272d876548'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:32.818550 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3e66e288-7c82-4022-a502-b1a182bfc047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c42c9aee1a249279e0856272d876548'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:32.819827 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3e66e288-7c82-4022-a502-b1a182bfc047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c42c9aee1a249279e0856272d876548'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:32.840792 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3e66e288-7c82-4022-a502-b1a182bfc047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c42c9aee1a249279e0856272d876548'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:32.852683 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3e66e288-7c82-4022-a502-b1a182bfc047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c42c9aee1a249279e0856272d876548'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:32.867477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 947/1899] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:32 2022] DELETE /identity/v3/users/3c42c9aee1a249279e0856272d876548 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:33.254238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:33.255892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:33.273962 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa576c-2abd-499c-b21d-7ab2273de61c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:33.354568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:33.355243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:33.355524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:33.355829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] PATH_INFO: `/v3/projects/03f4349bdacf46118078913950595b03` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:33.357156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03f4349bdacf46118078913950595b03)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:33.359108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:33.413967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:33.420056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03f4349bdacf46118078913950595b03', 'request_id': 'req-98374e42-8abb-4f00-b148-760cdee87c94'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.439813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03f4349bdacf46118078913950595b03', 'request_id': 'req-98374e42-8abb-4f00-b148-760cdee87c94'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.455580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03f4349bdacf46118078913950595b03', 'request_id': 'req-98374e42-8abb-4f00-b148-760cdee87c94'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.472155 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] The token cache is being invalidate because project 03f4349bdacf46118078913950595b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:33.472550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.473648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.475854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 953/1900] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:33 2022] DELETE /identity/v3/projects/03f4349bdacf46118078913950595b03 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:33.480388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4562c-6776-4669-a63c-f35f1f13250c None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:33.480966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4562c-6776-4669-a63c-f35f1f13250c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:33.480966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4562c-6776-4669-a63c-f35f1f13250c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:33.505086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4fe4562c-6776-4669-a63c-f35f1f13250c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:33.506104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4fe4562c-6776-4669-a63c-f35f1f13250c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:33.528329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:33.563916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:33.565364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:33.591503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 948/1901] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:33.600285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75482f81-6b8b-46e2-9259-377e69b3453d None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:33.600786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75482f81-6b8b-46e2-9259-377e69b3453d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:33.601571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75482f81-6b8b-46e2-9259-377e69b3453d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:33.642591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-75482f81-6b8b-46e2-9259-377e69b3453d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:33.644166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-75482f81-6b8b-46e2-9259-377e69b3453d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:33.658455 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 949/1902] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:33.659561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75482f81-6b8b-46e2-9259-377e69b3453d None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:33.660968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-75482f81-6b8b-46e2-9259-377e69b3453d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:33.664202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98374e42-8abb-4f00-b148-760cdee87c94 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:33.673673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:33.674326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:33.674603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:33.674876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] PATH_INFO: `/v3/users/8b7cd11cbbb1415487953523d8aaf822` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:33.676738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7cd11cbbb1415487953523d8aaf822)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:33.677436 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75482f81-6b8b-46e2-9259-377e69b3453d None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:33.678744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:33.683775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde68cb-4141-4842-8497-a8d9fcf832f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:33.684415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde68cb-4141-4842-8497-a8d9fcf832f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:33.684700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde68cb-4141-4842-8497-a8d9fcf832f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:33.684955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde68cb-4141-4842-8497-a8d9fcf832f6 admin admin] PATH_INFO: `/v3/users/abeb2acbe9e14adabb1d761f5dc89f1f` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:33.686313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde68cb-4141-4842-8497-a8d9fcf832f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abeb2acbe9e14adabb1d761f5dc89f1f)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:33.688228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde68cb-4141-4842-8497-a8d9fcf832f6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:33.738262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7cd11cbbb1415487953523d8aaf822'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.746575 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7cd11cbbb1415487953523d8aaf822'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.747303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7cd11cbbb1415487953523d8aaf822'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.748134 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7cd11cbbb1415487953523d8aaf822'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.753341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6dde68cb-4141-4842-8497-a8d9fcf832f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abeb2acbe9e14adabb1d761f5dc89f1f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.763421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7cd11cbbb1415487953523d8aaf822'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.768095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6dde68cb-4141-4842-8497-a8d9fcf832f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abeb2acbe9e14adabb1d761f5dc89f1f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.769145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6dde68cb-4141-4842-8497-a8d9fcf832f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abeb2acbe9e14adabb1d761f5dc89f1f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.782744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6dde68cb-4141-4842-8497-a8d9fcf832f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abeb2acbe9e14adabb1d761f5dc89f1f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.785076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 954/1903] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:33 2022] DELETE /identity/v3/users/8b7cd11cbbb1415487953523d8aaf822 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:33.786524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:33.789831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:33.799524 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6dde68cb-4141-4842-8497-a8d9fcf832f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abeb2acbe9e14adabb1d761f5dc89f1f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:33.806208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 950/1904] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:33 2022] DELETE /identity/v3/users/abeb2acbe9e14adabb1d761f5dc89f1f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:33.811120 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995de00-22ae-4544-8354-965b7597de34 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:33.811472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995de00-22ae-4544-8354-965b7597de34 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:33.811864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995de00-22ae-4544-8354-965b7597de34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:33.833670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3995de00-22ae-4544-8354-965b7597de34 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:33.834907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3995de00-22ae-4544-8354-965b7597de34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:33.850426 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:33.878974 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f44ab0-b68e-4bc0-9ac6-7013bb722e64 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:33.963965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 951/1905] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:33.963965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fbb87-d4b1-4b8e-b8e0-826005c2c9b3 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:33.963965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fbb87-d4b1-4b8e-b8e0-826005c2c9b3 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:33.963965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fbb87-d4b1-4b8e-b8e0-826005c2c9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:33.990417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add25fe9-9fd4-40a0-9dd2-e4a31dbab0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:33.991369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add25fe9-9fd4-40a0-9dd2-e4a31dbab0cb admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:33.991651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add25fe9-9fd4-40a0-9dd2-e4a31dbab0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:33.991964 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add25fe9-9fd4-40a0-9dd2-e4a31dbab0cb admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:33.992630 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add25fe9-9fd4-40a0-9dd2-e4a31dbab0cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:33.993024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add25fe9-9fd4-40a0-9dd2-e4a31dbab0cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:33.995411 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add25fe9-9fd4-40a0-9dd2-e4a31dbab0cb admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:33.996912 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-5d6fbb87-d4b1-4b8e-b8e0-826005c2c9b3 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:34.000115 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5d6fbb87-d4b1-4b8e-b8e0-826005c2c9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:34.014204 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 955/1906] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:34.015501 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add25fe9-9fd4-40a0-9dd2-e4a31dbab0cb admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:34.017545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-add25fe9-9fd4-40a0-9dd2-e4a31dbab0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:34.025193 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 952/1907] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:34.026187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fbb87-d4b1-4b8e-b8e0-826005c2c9b3 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:34.031404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5d6fbb87-d4b1-4b8e-b8e0-826005c2c9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:34.064984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fbb87-d4b1-4b8e-b8e0-826005c2c9b3 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:34.071956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1eef0-2c3d-43b9-9f31-9b9ec8f6ce0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:34.072805 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1eef0-2c3d-43b9-9f31-9b9ec8f6ce0d admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:34.073181 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1eef0-2c3d-43b9-9f31-9b9ec8f6ce0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:34.073595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1eef0-2c3d-43b9-9f31-9b9ec8f6ce0d admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:34.074390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1eef0-2c3d-43b9-9f31-9b9ec8f6ce0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:34.076674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1eef0-2c3d-43b9-9f31-9b9ec8f6ce0d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:34.128212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:34.147357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 953/1908] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:34 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:34.148056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1eef0-2c3d-43b9-9f31-9b9ec8f6ce0d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:34.149759 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-98e1eef0-2c3d-43b9-9f31-9b9ec8f6ce0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:34.170081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add25fe9-9fd4-40a0-9dd2-e4a31dbab0cb admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:34.261805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffc12d-eafb-4033-80df-8cd047a99a78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:34.263533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffc12d-eafb-4033-80df-8cd047a99a78 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:34.263874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffc12d-eafb-4033-80df-8cd047a99a78 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:34.264205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffc12d-eafb-4033-80df-8cd047a99a78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:34.293311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1eef0-2c3d-43b9-9f31-9b9ec8f6ce0d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:34.297825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8f7cf-bab8-4842-bd19-dd98d838f73b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:34.298973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8f7cf-bab8-4842-bd19-dd98d838f73b service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:34.299409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8f7cf-bab8-4842-bd19-dd98d838f73b service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:34.299886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8f7cf-bab8-4842-bd19-dd98d838f73b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:34.388887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:34.394625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ffc12d-eafb-4033-80df-8cd047a99a78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:34.396267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ffc12d-eafb-4033-80df-8cd047a99a78 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:34.411384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 956/1909] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:34.427229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffc12d-eafb-4033-80df-8cd047a99a78 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:34.440607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8f7cf-bab8-4842-bd19-dd98d838f73b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:34.442286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8f7cf-bab8-4842-bd19-dd98d838f73b service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:34.459542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffc12d-eafb-4033-80df-8cd047a99a78 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:34.470012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 954/1910] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:34.480294 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8acc91d-6456-496a-83f7-e42305fc51c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:34.481513 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8acc91d-6456-496a-83f7-e42305fc51c3 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:34.481943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8acc91d-6456-496a-83f7-e42305fc51c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:34.482506 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8acc91d-6456-496a-83f7-e42305fc51c3 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:34.485020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8acc91d-6456-496a-83f7-e42305fc51c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:34.488408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8acc91d-6456-496a-83f7-e42305fc51c3 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:34.539811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 957/1911] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:34 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:34.545911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8f7cf-bab8-4842-bd19-dd98d838f73b service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:34.579037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8f7cf-bab8-4842-bd19-dd98d838f73b service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:34.602947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23be13-9956-4f1d-973e-c0b2f1d5434d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:34.614455 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23be13-9956-4f1d-973e-c0b2f1d5434d admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:34.615826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23be13-9956-4f1d-973e-c0b2f1d5434d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:34.616348 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23be13-9956-4f1d-973e-c0b2f1d5434d admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:34.621695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23be13-9956-4f1d-973e-c0b2f1d5434d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:34.627548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23be13-9956-4f1d-973e-c0b2f1d5434d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:34.667487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 955/1912] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:34.672264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8acc91d-6456-496a-83f7-e42305fc51c3 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:34.692209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8acc91d-6456-496a-83f7-e42305fc51c3 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:34.715617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad5798-8e04-44ef-9c60-8b78410acae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:34.716818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad5798-8e04-44ef-9c60-8b78410acae0 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:34.717307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad5798-8e04-44ef-9c60-8b78410acae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:34.717833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad5798-8e04-44ef-9c60-8b78410acae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:34.719660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad5798-8e04-44ef-9c60-8b78410acae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:34.723999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad5798-8e04-44ef-9c60-8b78410acae0 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:34.751420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 958/1913] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:34.766122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23be13-9956-4f1d-973e-c0b2f1d5434d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:34.785400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23be13-9956-4f1d-973e-c0b2f1d5434d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:34.797888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a5dc2-31b5-482c-acb3-221ead218cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:34.798587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a5dc2-31b5-482c-acb3-221ead218cf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:34.798871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a5dc2-31b5-482c-acb3-221ead218cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:34.799154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a5dc2-31b5-482c-acb3-221ead218cf1 admin admin] PATH_INFO: `/v3/projects/5197c15b6f1145989cec0f17a1832c82/users/37fc73891a324413a06e1205a1082745/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:34.823812 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a5dc2-31b5-482c-acb3-221ead218cf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5197c15b6f1145989cec0f17a1832c82, user_id=37fc73891a324413a06e1205a1082745, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:34.830631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a5dc2-31b5-482c-acb3-221ead218cf1 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:34.860575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 956/1914] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:34 2022] PUT /identity/v3/projects/5197c15b6f1145989cec0f17a1832c82/users/37fc73891a324413a06e1205a1082745/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:34.872070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2fa08-fa47-4e92-9a80-6be94b8237ce None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:34.872951 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2fa08-fa47-4e92-9a80-6be94b8237ce None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:34.873457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2fa08-fa47-4e92-9a80-6be94b8237ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:34.920366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-0cb2fa08-fa47-4e92-9a80-6be94b8237ce None None] MFA Rules not processed for user `37fc73891a324413a06e1205a1082745`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:34.921924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-0cb2fa08-fa47-4e92-9a80-6be94b8237ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:35.019065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 959/1915] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:35.024355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a5dc2-31b5-482c-acb3-221ead218cf1 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:35.101794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2fa08-fa47-4e92-9a80-6be94b8237ce None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:35.103303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-0cb2fa08-fa47-4e92-9a80-6be94b8237ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:35.128415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a5dc2-31b5-482c-acb3-221ead218cf1 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:35.135789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e8490-6d73-4554-aacc-1407603d187b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:35.135789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e8490-6d73-4554-aacc-1407603d187b admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:35.135789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e8490-6d73-4554-aacc-1407603d187b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:35.135789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e8490-6d73-4554-aacc-1407603d187b admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:35.136986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e8490-6d73-4554-aacc-1407603d187b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:35.139462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e8490-6d73-4554-aacc-1407603d187b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:35.177894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:35.192163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 957/1916] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:35 2022] POST /identity/v3/projects => generated 365 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:35.196066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e8490-6d73-4554-aacc-1407603d187b admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:35.220578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2fa08-fa47-4e92-9a80-6be94b8237ce None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:35.289664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e8490-6d73-4554-aacc-1407603d187b admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:35.298595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc4ec8-8259-481b-a3eb-5f416058d5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:35.299454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc4ec8-8259-481b-a3eb-5f416058d5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:35.299920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc4ec8-8259-481b-a3eb-5f416058d5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:35.300373 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc4ec8-8259-481b-a3eb-5f416058d5a4 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:35.302852 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc4ec8-8259-481b-a3eb-5f416058d5a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:35.306018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc4ec8-8259-481b-a3eb-5f416058d5a4 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:35.341503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:35.342209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:35.342560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:35.343002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] PATH_INFO: `/v3/projects/76284b3bb22849e4a47df986926bd86a` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:35.344364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=76284b3bb22849e4a47df986926bd86a)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:35.346540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:35.350252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:35.354437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 958/1917] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:35 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:35.354437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc4ec8-8259-481b-a3eb-5f416058d5a4 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:35.355119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ecbc4ec8-8259-481b-a3eb-5f416058d5a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:35.384084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc4ec8-8259-481b-a3eb-5f416058d5a4 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:35.392565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c588f6-572d-4ef1-9c02-95a1fc86ec18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:35.394345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c588f6-572d-4ef1-9c02-95a1fc86ec18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:35.394345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c588f6-572d-4ef1-9c02-95a1fc86ec18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:35.394612 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c588f6-572d-4ef1-9c02-95a1fc86ec18 admin admin] PATH_INFO: `/v3/projects/fb84e10ce5534a68b4a36b1bcb2eef0f` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:35.396514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c588f6-572d-4ef1-9c02-95a1fc86ec18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb84e10ce5534a68b4a36b1bcb2eef0f)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:35.398416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c588f6-572d-4ef1-9c02-95a1fc86ec18 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:35.457234 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76284b3bb22849e4a47df986926bd86a', 'request_id': 'req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:35.473570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76284b3bb22849e4a47df986926bd86a', 'request_id': 'req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:35.489829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76284b3bb22849e4a47df986926bd86a', 'request_id': 'req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:35.515308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] The token cache is being invalidate because project 76284b3bb22849e4a47df986926bd86a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:35.515609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:35.520291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:35.526113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 960/1918] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:35 2022] DELETE /identity/v3/projects/76284b3bb22849e4a47df986926bd86a => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:35.527099 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:35.528640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:35.543755 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-52c588f6-572d-4ef1-9c02-95a1fc86ec18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb84e10ce5534a68b4a36b1bcb2eef0f', 'request_id': 'req-52c588f6-572d-4ef1-9c02-95a1fc86ec18'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:35.559469 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-52c588f6-572d-4ef1-9c02-95a1fc86ec18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb84e10ce5534a68b4a36b1bcb2eef0f', 'request_id': 'req-52c588f6-572d-4ef1-9c02-95a1fc86ec18'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:35.578584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-52c588f6-572d-4ef1-9c02-95a1fc86ec18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb84e10ce5534a68b4a36b1bcb2eef0f', 'request_id': 'req-52c588f6-572d-4ef1-9c02-95a1fc86ec18'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:35.625460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-52c588f6-572d-4ef1-9c02-95a1fc86ec18 admin admin] The token cache is being invalidate because project fb84e10ce5534a68b4a36b1bcb2eef0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:35.625869 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-52c588f6-572d-4ef1-9c02-95a1fc86ec18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:35.633051 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-52c588f6-572d-4ef1-9c02-95a1fc86ec18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:35.644900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 959/1919] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:35 2022] DELETE /identity/v3/projects/fb84e10ce5534a68b4a36b1bcb2eef0f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:35.646925 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a694ad-c743-4648-aacc-90832dd3c88c None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:35.647211 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a694ad-c743-4648-aacc-90832dd3c88c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:35.647486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a694ad-c743-4648-aacc-90832dd3c88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:35.693947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-a4a694ad-c743-4648-aacc-90832dd3c88c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:35.695987 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a4a694ad-c743-4648-aacc-90832dd3c88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:35.743161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438fdd7-8cbc-4214-bfaf-ddca4b5a3ceb admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:35.821212 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e987487-e718-4ad1-8120-e8c2982d0086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:35.821850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e987487-e718-4ad1-8120-e8c2982d0086 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:35.822129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e987487-e718-4ad1-8120-e8c2982d0086 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:35.822399 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e987487-e718-4ad1-8120-e8c2982d0086 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:35.823417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e987487-e718-4ad1-8120-e8c2982d0086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:35.825681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e987487-e718-4ad1-8120-e8c2982d0086 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:35.836529 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 961/1920] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:35.837423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e987487-e718-4ad1-8120-e8c2982d0086 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:35.850210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9e987487-e718-4ad1-8120-e8c2982d0086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:35.881650 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e987487-e718-4ad1-8120-e8c2982d0086 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:35.888479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 960/1921] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:35.895463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a694ad-c743-4648-aacc-90832dd3c88c None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:35.897036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a4a694ad-c743-4648-aacc-90832dd3c88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:35.907958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:35.910834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:35.912873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:35.913469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] PATH_INFO: `/v3/users/233b973f398341c7b27d42beae4b21b1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:35.931213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=233b973f398341c7b27d42beae4b21b1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:35.932129 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a694ad-c743-4648-aacc-90832dd3c88c None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:35.933944 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:35.939017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258ddd2-3f65-4e4c-a508-60108ab6c511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:35.939849 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258ddd2-3f65-4e4c-a508-60108ab6c511 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:35.940343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258ddd2-3f65-4e4c-a508-60108ab6c511 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:35.940859 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258ddd2-3f65-4e4c-a508-60108ab6c511 admin admin] PATH_INFO: `/v3/projects/bd2f83b76f5444caa020bbc2a77b4bd3/users/f64ce609fde5497798d5bad22b44de04/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:35.966639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:35.977208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258ddd2-3f65-4e4c-a508-60108ab6c511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd2f83b76f5444caa020bbc2a77b4bd3, user_id=f64ce609fde5497798d5bad22b44de04, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:35.986209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258ddd2-3f65-4e4c-a508-60108ab6c511 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:36.006949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:36.012277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 961/1922] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:35 2022] PUT /identity/v3/projects/bd2f83b76f5444caa020bbc2a77b4bd3/users/f64ce609fde5497798d5bad22b44de04/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:36.022166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec249c62-d92c-4c11-bc1c-16ad54e18c4c None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:36.022778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec249c62-d92c-4c11-bc1c-16ad54e18c4c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:36.023205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec249c62-d92c-4c11-bc1c-16ad54e18c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:36.034341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '233b973f398341c7b27d42beae4b21b1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:36.047206 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '233b973f398341c7b27d42beae4b21b1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:36.047994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '233b973f398341c7b27d42beae4b21b1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:36.048782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '233b973f398341c7b27d42beae4b21b1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:36.068282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '233b973f398341c7b27d42beae4b21b1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:36.088044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-ec249c62-d92c-4c11-bc1c-16ad54e18c4c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:36.088822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 962/1923] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:35 2022] DELETE /identity/v3/users/233b973f398341c7b27d42beae4b21b1 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:36.093109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:36.093332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ec249c62-d92c-4c11-bc1c-16ad54e18c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:36.093616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:36.228038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 962/1924] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:36.233446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec249c62-d92c-4c11-bc1c-16ad54e18c4c None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:36.250465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec249c62-d92c-4c11-bc1c-16ad54e18c4c None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:36.254900 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3459590-e340-401b-8e97-5ce6f2d0bba1 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:36.256804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3244c-eb1d-4c09-b6a0-7600c25c2cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:36.257746 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3244c-eb1d-4c09-b6a0-7600c25c2cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:36.258208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3244c-eb1d-4c09-b6a0-7600c25c2cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:36.258486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3244c-eb1d-4c09-b6a0-7600c25c2cd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:36.259141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3244c-eb1d-4c09-b6a0-7600c25c2cd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:36.259440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3244c-eb1d-4c09-b6a0-7600c25c2cd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:36.262619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3244c-eb1d-4c09-b6a0-7600c25c2cd4 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:36.265533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d31a6f-3a10-4a12-b86b-9ff7c93e6ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:36.266958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d31a6f-3a10-4a12-b86b-9ff7c93e6ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:36.267422 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d31a6f-3a10-4a12-b86b-9ff7c93e6ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:36.267973 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d31a6f-3a10-4a12-b86b-9ff7c93e6ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:36.269383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d31a6f-3a10-4a12-b86b-9ff7c93e6ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:36.274068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d31a6f-3a10-4a12-b86b-9ff7c93e6ba5 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:36.280546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 963/1925] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:36.288259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3244c-eb1d-4c09-b6a0-7600c25c2cd4 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:36.303597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 963/1926] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:36.306347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d31a6f-3a10-4a12-b86b-9ff7c93e6ba5 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:36.311980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3244c-eb1d-4c09-b6a0-7600c25c2cd4 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:36.317462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b87d9d-bfe1-4518-8a55-d8d927323a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:36.318693 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b87d9d-bfe1-4518-8a55-d8d927323a96 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:36.318996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b87d9d-bfe1-4518-8a55-d8d927323a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:36.319273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b87d9d-bfe1-4518-8a55-d8d927323a96 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:36.320311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b87d9d-bfe1-4518-8a55-d8d927323a96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:36.324797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b87d9d-bfe1-4518-8a55-d8d927323a96 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:36.332831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d31a6f-3a10-4a12-b86b-9ff7c93e6ba5 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:36.347648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e6f99-8524-4f43-bd6c-42493bc7ae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:36.348466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e6f99-8524-4f43-bd6c-42493bc7ae2f admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:36.348747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e6f99-8524-4f43-bd6c-42493bc7ae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:36.349014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e6f99-8524-4f43-bd6c-42493bc7ae2f admin admin] PATH_INFO: `/v3/projects/bd2f83b76f5444caa020bbc2a77b4bd3/users/f64ce609fde5497798d5bad22b44de04/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:36.354809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e6f99-8524-4f43-bd6c-42493bc7ae2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd2f83b76f5444caa020bbc2a77b4bd3, user_id=f64ce609fde5497798d5bad22b44de04, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:36.363091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e6f99-8524-4f43-bd6c-42493bc7ae2f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:36.403035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 964/1927] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:36 2022] PUT /identity/v3/projects/bd2f83b76f5444caa020bbc2a77b4bd3/users/f64ce609fde5497798d5bad22b44de04/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:36.406234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 964/1928] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:36 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:36.410546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e6f99-8524-4f43-bd6c-42493bc7ae2f admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:36.414771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfa924-0be0-41a3-b841-de88bea849c7 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:36.415314 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfa924-0be0-41a3-b841-de88bea849c7 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:36.415669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfa924-0be0-41a3-b841-de88bea849c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:36.499315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e6f99-8524-4f43-bd6c-42493bc7ae2f admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:36.506956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-cddfa924-0be0-41a3-b841-de88bea849c7 None None] MFA Rules not processed for user `f64ce609fde5497798d5bad22b44de04`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:36.511516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-cddfa924-0be0-41a3-b841-de88bea849c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:36.562822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:36.601656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae23482-2f06-4581-9158-072696fdecd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:36.603218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae23482-2f06-4581-9158-072696fdecd0 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:36.604460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae23482-2f06-4581-9158-072696fdecd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:36.605039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae23482-2f06-4581-9158-072696fdecd0 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:36.605846 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae23482-2f06-4581-9158-072696fdecd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:36.607979 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae23482-2f06-4581-9158-072696fdecd0 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:36.673401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 965/1929] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:36 2022] POST /identity/v3/users => generated 282 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:36.674393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae23482-2f06-4581-9158-072696fdecd0 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:36.695022 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae23482-2f06-4581-9158-072696fdecd0 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:36.719763 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a22dc-a046-426d-a671-8db478756512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:36.720404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a22dc-a046-426d-a671-8db478756512 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:36.720702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a22dc-a046-426d-a671-8db478756512 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:36.721530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a22dc-a046-426d-a671-8db478756512 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:36.722342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a22dc-a046-426d-a671-8db478756512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:36.723822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a22dc-a046-426d-a671-8db478756512 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:36.724030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 965/1930] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:36.733372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfa924-0be0-41a3-b841-de88bea849c7 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:36.735259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-cddfa924-0be0-41a3-b841-de88bea849c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:36.752485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 966/1931] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:36.754340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a22dc-a046-426d-a671-8db478756512 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:36.768701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a22dc-a046-426d-a671-8db478756512 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:36.774205 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7019487-7df9-40ad-92bc-7762f6f2ee4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:36.774916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7019487-7df9-40ad-92bc-7762f6f2ee4b admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:36.775283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7019487-7df9-40ad-92bc-7762f6f2ee4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:36.775753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7019487-7df9-40ad-92bc-7762f6f2ee4b admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:36.776515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7019487-7df9-40ad-92bc-7762f6f2ee4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:36.779626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7019487-7df9-40ad-92bc-7762f6f2ee4b admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:36.797606 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 967/1932] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:36.802893 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7019487-7df9-40ad-92bc-7762f6f2ee4b admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:36.819804 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7019487-7df9-40ad-92bc-7762f6f2ee4b admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:36.828397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75745341-850f-4d47-8864-f861cf0f3018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:36.829214 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75745341-850f-4d47-8864-f861cf0f3018 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:36.829559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75745341-850f-4d47-8864-f861cf0f3018 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:36.829901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75745341-850f-4d47-8864-f861cf0f3018 admin admin] PATH_INFO: `/v3/projects/a2c6091e7a344c4f9e53cc1544466daf/users/91e84988e6284c3a9760396c79e2269d/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:36.849767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:36.876812 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfa924-0be0-41a3-b841-de88bea849c7 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:36.880276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75745341-850f-4d47-8864-f861cf0f3018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2c6091e7a344c4f9e53cc1544466daf, user_id=91e84988e6284c3a9760396c79e2269d, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:36.888617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc43a5-a87c-4e8d-9e86-a5f3b7e322eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:36.889272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc43a5-a87c-4e8d-9e86-a5f3b7e322eb service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:36.889560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc43a5-a87c-4e8d-9e86-a5f3b7e322eb service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:36.889821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc43a5-a87c-4e8d-9e86-a5f3b7e322eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:36.900713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75745341-850f-4d47-8864-f861cf0f3018 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:36.912311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc43a5-a87c-4e8d-9e86-a5f3b7e322eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:36.917522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc43a5-a87c-4e8d-9e86-a5f3b7e322eb service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:36.927362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 966/1933] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:36.928671 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc43a5-a87c-4e8d-9e86-a5f3b7e322eb service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:36.929797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3fcc43a5-a87c-4e8d-9e86-a5f3b7e322eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:36.943773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 968/1934] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:36 2022] PUT /identity/v3/projects/a2c6091e7a344c4f9e53cc1544466daf/users/91e84988e6284c3a9760396c79e2269d/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:36.951491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b3948-fcd2-462b-bdcd-8adb792fe6d2 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:36.951808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b3948-fcd2-462b-bdcd-8adb792fe6d2 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:36.952087 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b3948-fcd2-462b-bdcd-8adb792fe6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:36.986931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc43a5-a87c-4e8d-9e86-a5f3b7e322eb service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:37.037822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-144b3948-fcd2-462b-bdcd-8adb792fe6d2 None None] MFA Rules not processed for user `91e84988e6284c3a9760396c79e2269d`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:37.039134 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-144b3948-fcd2-462b-bdcd-8adb792fe6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:37.129229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:37.129903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:37.130362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:37.131743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] PATH_INFO: `/v3/projects/c40c9d4272c34cb9b08e5e42af4db95a` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:37.134960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c40c9d4272c34cb9b08e5e42af4db95a)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:37.142356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:37.176262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 969/1935] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:37.176262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b3948-fcd2-462b-bdcd-8adb792fe6d2 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:37.263658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b3948-fcd2-462b-bdcd-8adb792fe6d2 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:37.272339 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c40c9d4272c34cb9b08e5e42af4db95a', 'request_id': 'req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:37.288399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c40c9d4272c34cb9b08e5e42af4db95a', 'request_id': 'req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:37.303372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c40c9d4272c34cb9b08e5e42af4db95a', 'request_id': 'req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:37.320262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] The token cache is being invalidate because project c40c9d4272c34cb9b08e5e42af4db95a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:37.320262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:37.320262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:37.331330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:37.332900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 967/1936] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:36 2022] DELETE /identity/v3/projects/c40c9d4272c34cb9b08e5e42af4db95a => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:37.336384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:37.348928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:37.405425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7405362-7bf7-4f64-a7d7-425c50d6209f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:37.406145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7405362-7bf7-4f64-a7d7-425c50d6209f service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:37.407797 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7405362-7bf7-4f64-a7d7-425c50d6209f service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:37.408255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7405362-7bf7-4f64-a7d7-425c50d6209f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:37.416015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c7405362-7bf7-4f64-a7d7-425c50d6209f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:37.497147 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6eaa19-0628-4fed-add7-f6b2f8d6501a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:37.501917 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4cd2f-5e29-4f61-986c-7525e78b33b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:37.502718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4cd2f-5e29-4f61-986c-7525e78b33b8 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:37.503157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4cd2f-5e29-4f61-986c-7525e78b33b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:37.503534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4cd2f-5e29-4f61-986c-7525e78b33b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:37.505741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b1e4cd2f-5e29-4f61-986c-7525e78b33b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:37.511471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7405362-7bf7-4f64-a7d7-425c50d6209f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:37.513189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7405362-7bf7-4f64-a7d7-425c50d6209f service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:37.527654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 970/1937] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:37.529012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c3ad1-09d1-4ac9-8490-102daa8362ec None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:37.529299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c3ad1-09d1-4ac9-8490-102daa8362ec None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:37.529703 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c3ad1-09d1-4ac9-8490-102daa8362ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:37.552742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-c52c3ad1-09d1-4ac9-8490-102daa8362ec None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:37.554010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c52c3ad1-09d1-4ac9-8490-102daa8362ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:37.578289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:37.650204 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4cd2f-5e29-4f61-986c-7525e78b33b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:37.652517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4cd2f-5e29-4f61-986c-7525e78b33b8 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:37.661518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 971/1938] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:37.667919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 968/1939] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:37.671560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a950630-b096-416a-8a81-eee2a1dd97a1 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:37.672638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a950630-b096-416a-8a81-eee2a1dd97a1 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:37.673460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a950630-b096-416a-8a81-eee2a1dd97a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:37.702444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-9a950630-b096-416a-8a81-eee2a1dd97a1 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:37.703989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9a950630-b096-416a-8a81-eee2a1dd97a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:37.717608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 969/1940] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:37.722282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c3ad1-09d1-4ac9-8490-102daa8362ec None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:37.734641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c3ad1-09d1-4ac9-8490-102daa8362ec None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:37.740142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e79cf-2c36-4c1f-b940-5c1aa973ed68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:37.740931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e79cf-2c36-4c1f-b940-5c1aa973ed68 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:37.741220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e79cf-2c36-4c1f-b940-5c1aa973ed68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:37.741499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e79cf-2c36-4c1f-b940-5c1aa973ed68 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:37.742171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e79cf-2c36-4c1f-b940-5c1aa973ed68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:37.742482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e79cf-2c36-4c1f-b940-5c1aa973ed68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:37.744347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e79cf-2c36-4c1f-b940-5c1aa973ed68 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:37.771069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 972/1941] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:37.772796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a950630-b096-416a-8a81-eee2a1dd97a1 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:37.788626 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a950630-b096-416a-8a81-eee2a1dd97a1 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:37.797559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a223c-3aad-4d62-8991-1f537108e889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:37.799232 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a223c-3aad-4d62-8991-1f537108e889 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:37.799796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a223c-3aad-4d62-8991-1f537108e889 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:37.800363 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a223c-3aad-4d62-8991-1f537108e889 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:37.802152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a223c-3aad-4d62-8991-1f537108e889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:37.818305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a223c-3aad-4d62-8991-1f537108e889 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:37.878846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 970/1942] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:37 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:37.884783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e79cf-2c36-4c1f-b940-5c1aa973ed68 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:37.913556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e79cf-2c36-4c1f-b940-5c1aa973ed68 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:38.025821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a223c-3aad-4d62-8991-1f537108e889 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:38.027739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b56a223c-3aad-4d62-8991-1f537108e889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:38.076643 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386ec1b-b851-4404-a862-88bf3acab2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:38.077528 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386ec1b-b851-4404-a862-88bf3acab2fe admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:38.077880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386ec1b-b851-4404-a862-88bf3acab2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:38.078228 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386ec1b-b851-4404-a862-88bf3acab2fe admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:38.078961 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386ec1b-b851-4404-a862-88bf3acab2fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:38.081379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386ec1b-b851-4404-a862-88bf3acab2fe admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:38.107908 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:38.120699 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 973/1943] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:37 2022] POST /identity/v3/users => generated 285 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:38.124848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386ec1b-b851-4404-a862-88bf3acab2fe admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:38.129720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a223c-3aad-4d62-8991-1f537108e889 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:38.136318 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9279bd2-1890-4ca9-90ba-8aa0ff3d0401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:38.136891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9279bd2-1890-4ca9-90ba-8aa0ff3d0401 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:38.137195 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9279bd2-1890-4ca9-90ba-8aa0ff3d0401 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:38.137438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9279bd2-1890-4ca9-90ba-8aa0ff3d0401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:38.139293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b9279bd2-1890-4ca9-90ba-8aa0ff3d0401 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:38.141484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386ec1b-b851-4404-a862-88bf3acab2fe admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:38.148892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62276633-d07b-43b1-aaa9-f8a2f309f1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:38.149537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62276633-d07b-43b1-aaa9-f8a2f309f1fc admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:38.149948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62276633-d07b-43b1-aaa9-f8a2f309f1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:38.150232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62276633-d07b-43b1-aaa9-f8a2f309f1fc admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:38.151307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62276633-d07b-43b1-aaa9-f8a2f309f1fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:38.153149 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62276633-d07b-43b1-aaa9-f8a2f309f1fc admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:38.166983 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 974/1944] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:38.171171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62276633-d07b-43b1-aaa9-f8a2f309f1fc admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:38.195235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62276633-d07b-43b1-aaa9-f8a2f309f1fc admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:38.208113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27866221-56c4-4523-b3e7-ff9011a39fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:38.208888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27866221-56c4-4523-b3e7-ff9011a39fcd admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:38.209232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27866221-56c4-4523-b3e7-ff9011a39fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:38.209921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27866221-56c4-4523-b3e7-ff9011a39fcd admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:38.210287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27866221-56c4-4523-b3e7-ff9011a39fcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:38.212539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27866221-56c4-4523-b3e7-ff9011a39fcd admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:38.225015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 975/1945] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:38.229525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27866221-56c4-4523-b3e7-ff9011a39fcd admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:38.232842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9279bd2-1890-4ca9-90ba-8aa0ff3d0401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:38.234951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9279bd2-1890-4ca9-90ba-8aa0ff3d0401 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:38.242484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27866221-56c4-4523-b3e7-ff9011a39fcd admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:38.262331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f5f81-c6fc-474f-b3d1-5a14d624fc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:38.263166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f5f81-c6fc-474f-b3d1-5a14d624fc3d admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:38.263514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f5f81-c6fc-474f-b3d1-5a14d624fc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:38.264004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f5f81-c6fc-474f-b3d1-5a14d624fc3d admin admin] PATH_INFO: `/v3/projects/10d7741ed2f14136b7865fac9d3371dc/users/c0d9ab94cf6846e7bde6eb86b40dd29d/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:38.271262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 971/1946] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:38.302204 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f5f81-c6fc-474f-b3d1-5a14d624fc3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d7741ed2f14136b7865fac9d3371dc, user_id=c0d9ab94cf6846e7bde6eb86b40dd29d, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:38.311989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f5f81-c6fc-474f-b3d1-5a14d624fc3d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:38.334702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 976/1947] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:38 2022] PUT /identity/v3/projects/10d7741ed2f14136b7865fac9d3371dc/users/c0d9ab94cf6846e7bde6eb86b40dd29d/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:38.347042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1630-cde3-410c-9215-1953c6e4d83a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:38.347717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1630-cde3-410c-9215-1953c6e4d83a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:38.348388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1630-cde3-410c-9215-1953c6e4d83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:38.398799 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-c3ca1630-cde3-410c-9215-1953c6e4d83a None None] MFA Rules not processed for user `c0d9ab94cf6846e7bde6eb86b40dd29d`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:38.400563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c3ca1630-cde3-410c-9215-1953c6e4d83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:38.485444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:38.537262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 972/1948] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:38.542265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f5f81-c6fc-474f-b3d1-5a14d624fc3d admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:38.671898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f5f81-c6fc-474f-b3d1-5a14d624fc3d admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:38.678279 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9b92c-e5cc-45d4-99db-29c9e8072167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:38.679096 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9b92c-e5cc-45d4-99db-29c9e8072167 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:38.679453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9b92c-e5cc-45d4-99db-29c9e8072167 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:38.679894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9b92c-e5cc-45d4-99db-29c9e8072167 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:38.680664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9b92c-e5cc-45d4-99db-29c9e8072167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:38.684129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9b92c-e5cc-45d4-99db-29c9e8072167 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:38.741383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 977/1949] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:38 2022] POST /identity/v3/projects => generated 367 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:38.745512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca1630-cde3-410c-9215-1953c6e4d83a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:38.757897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca1630-cde3-410c-9215-1953c6e4d83a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:38.889602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807ccca-ab30-4ae7-9390-08ce52b97a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:38.894162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807ccca-ab30-4ae7-9390-08ce52b97a75 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:38.894614 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807ccca-ab30-4ae7-9390-08ce52b97a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:38.894924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807ccca-ab30-4ae7-9390-08ce52b97a75 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:38.896062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807ccca-ab30-4ae7-9390-08ce52b97a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:38.898920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807ccca-ab30-4ae7-9390-08ce52b97a75 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:38.950620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9b92c-e5cc-45d4-99db-29c9e8072167 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:38.967308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9b92c-e5cc-45d4-99db-29c9e8072167 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:38.971135 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 973/1950] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:38 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:38.974681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b0098-9d25-40c9-b333-73bf1f3be584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:38.975407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b0098-9d25-40c9-b333-73bf1f3be584 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:38.976430 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b0098-9d25-40c9-b333-73bf1f3be584 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:38.976892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b0098-9d25-40c9-b333-73bf1f3be584 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:38.977806 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b0098-9d25-40c9-b333-73bf1f3be584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:38.987477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b0098-9d25-40c9-b333-73bf1f3be584 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:39.013524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 978/1951] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:39.018006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807ccca-ab30-4ae7-9390-08ce52b97a75 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:39.046634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807ccca-ab30-4ae7-9390-08ce52b97a75 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:39.054012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bb7b8-474b-4a4f-be93-0cdab719fc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:39.054838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bb7b8-474b-4a4f-be93-0cdab719fc36 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:39.055185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bb7b8-474b-4a4f-be93-0cdab719fc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:39.055528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bb7b8-474b-4a4f-be93-0cdab719fc36 admin admin] PATH_INFO: `/v3/projects/9710621d62ec484385b2666731ae6c4e/users/abfedf7b5d9641bb8472d60fb22a44c2/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:39.079874 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bb7b8-474b-4a4f-be93-0cdab719fc36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9710621d62ec484385b2666731ae6c4e, user_id=abfedf7b5d9641bb8472d60fb22a44c2, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:39.084079 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bb7b8-474b-4a4f-be93-0cdab719fc36 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:39.110900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 974/1952] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:39 2022] PUT /identity/v3/projects/9710621d62ec484385b2666731ae6c4e/users/abfedf7b5d9641bb8472d60fb22a44c2/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:39.115415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b0098-9d25-40c9-b333-73bf1f3be584 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:39.138802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b0098-9d25-40c9-b333-73bf1f3be584 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:39.229044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb53ce0-3161-4149-b383-1b3374447e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:39.230067 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb53ce0-3161-4149-b383-1b3374447e41 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:39.230631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb53ce0-3161-4149-b383-1b3374447e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:39.230631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb53ce0-3161-4149-b383-1b3374447e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:39.233943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb53ce0-3161-4149-b383-1b3374447e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:39.233943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb53ce0-3161-4149-b383-1b3374447e41 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:39.245223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 979/1953] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:39.249334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bb7b8-474b-4a4f-be93-0cdab719fc36 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:39.262983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bb7b8-474b-4a4f-be93-0cdab719fc36 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:39.267880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b6d04-7257-44d6-9de2-f9f88fe9d1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:39.268557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b6d04-7257-44d6-9de2-f9f88fe9d1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:39.268835 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b6d04-7257-44d6-9de2-f9f88fe9d1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:39.269107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b6d04-7257-44d6-9de2-f9f88fe9d1b0 admin admin] PATH_INFO: `/v3/projects/9710621d62ec484385b2666731ae6c4e/users/abfedf7b5d9641bb8472d60fb22a44c2/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:39.271012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b6d04-7257-44d6-9de2-f9f88fe9d1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9710621d62ec484385b2666731ae6c4e, user_id=abfedf7b5d9641bb8472d60fb22a44c2, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:39.273952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b6d04-7257-44d6-9de2-f9f88fe9d1b0 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:39.287919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 975/1954] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:39 2022] PUT /identity/v3/projects/9710621d62ec484385b2666731ae6c4e/users/abfedf7b5d9641bb8472d60fb22a44c2/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:39.298387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bbc64-410f-44e4-8a6b-feaf8bc1a6c8 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:39.298668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bbc64-410f-44e4-8a6b-feaf8bc1a6c8 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:39.298940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bbc64-410f-44e4-8a6b-feaf8bc1a6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:39.373040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-9c6bbc64-410f-44e4-8a6b-feaf8bc1a6c8 None None] MFA Rules not processed for user `abfedf7b5d9641bb8472d60fb22a44c2`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:39.375373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9c6bbc64-410f-44e4-8a6b-feaf8bc1a6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:39.408664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:39.576539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 980/1955] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:39.627184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b6d04-7257-44d6-9de2-f9f88fe9d1b0 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:39.753804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b6d04-7257-44d6-9de2-f9f88fe9d1b0 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:39.764639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700dde9-aa54-4e0d-a6c0-df1ad2ecb06a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:39.765529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700dde9-aa54-4e0d-a6c0-df1ad2ecb06a service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:39.765811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700dde9-aa54-4e0d-a6c0-df1ad2ecb06a service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:39.766321 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700dde9-aa54-4e0d-a6c0-df1ad2ecb06a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:39.809840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700dde9-aa54-4e0d-a6c0-df1ad2ecb06a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:39.814332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700dde9-aa54-4e0d-a6c0-df1ad2ecb06a service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:39.834836 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 976/1956] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:39.920417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bbc64-410f-44e4-8a6b-feaf8bc1a6c8 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:39.935808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bbc64-410f-44e4-8a6b-feaf8bc1a6c8 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:39.939772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7b9c9-66d6-499d-8e30-5070263d320e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:39.941433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7b9c9-66d6-499d-8e30-5070263d320e service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:39.941935 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7b9c9-66d6-499d-8e30-5070263d320e service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:39.942411 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7b9c9-66d6-499d-8e30-5070263d320e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:40.021881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7b9c9-66d6-499d-8e30-5070263d320e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:40.024276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7b9c9-66d6-499d-8e30-5070263d320e service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:40.039551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 981/1957] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:40.587327 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700dde9-aa54-4e0d-a6c0-df1ad2ecb06a service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:40.588821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b700dde9-aa54-4e0d-a6c0-df1ad2ecb06a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:40.650130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:40.716101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700dde9-aa54-4e0d-a6c0-df1ad2ecb06a service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:40.716101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:40.716101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:40.716101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:40.716515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] PATH_INFO: `/v3/users/37fc73891a324413a06e1205a1082745` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:40.716515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=37fc73891a324413a06e1205a1082745)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:40.716515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:40.782625 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37fc73891a324413a06e1205a1082745'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:40.790707 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37fc73891a324413a06e1205a1082745'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:40.792373 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37fc73891a324413a06e1205a1082745'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:40.807719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37fc73891a324413a06e1205a1082745'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:40.819972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37fc73891a324413a06e1205a1082745'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:40.836586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 977/1958] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:40 2022] DELETE /identity/v3/users/37fc73891a324413a06e1205a1082745 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:40.839243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceddd1-f947-4723-ae46-5c5622c2d322 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:40.839849 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceddd1-f947-4723-ae46-5c5622c2d322 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:40.839941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceddd1-f947-4723-ae46-5c5622c2d322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:40.870360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-78ceddd1-f947-4723-ae46-5c5622c2d322 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:40.871636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-78ceddd1-f947-4723-ae46-5c5622c2d322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:40.987213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 982/1959] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:40.998280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:41.000557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:41.088986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65ab31-0a29-402b-b1e8-aec714527dfe admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:41.094514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d63ab4-a60d-41df-becd-b81fbe5472ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:41.095493 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d63ab4-a60d-41df-becd-b81fbe5472ca service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:41.095811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d63ab4-a60d-41df-becd-b81fbe5472ca service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:41.096086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d63ab4-a60d-41df-becd-b81fbe5472ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:41.114653 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d63ab4-a60d-41df-becd-b81fbe5472ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:41.116292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d63ab4-a60d-41df-becd-b81fbe5472ca service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:41.128505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 978/1960] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:41.234685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceddd1-f947-4723-ae46-5c5622c2d322 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:41.236312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-78ceddd1-f947-4723-ae46-5c5622c2d322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:41.251292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceddd1-f947-4723-ae46-5c5622c2d322 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:41.256683 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:41.257629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:41.257916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:41.258183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] PATH_INFO: `/v3/users/be429f537d0849d98e4d9b8f6ac1bc30` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:41.259379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be429f537d0849d98e4d9b8f6ac1bc30)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:41.261440 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:41.343368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:41.356339 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be429f537d0849d98e4d9b8f6ac1bc30'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:41.396518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be429f537d0849d98e4d9b8f6ac1bc30'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:41.398431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be429f537d0849d98e4d9b8f6ac1bc30'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:41.400252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be429f537d0849d98e4d9b8f6ac1bc30'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:41.431076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be429f537d0849d98e4d9b8f6ac1bc30'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:41.458109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 983/1961] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:41 2022] DELETE /identity/v3/users/be429f537d0849d98e4d9b8f6ac1bc30 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:41.466205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ac213-dfa6-46ef-8bce-4edcb45f3b7f None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:41.466756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ac213-dfa6-46ef-8bce-4edcb45f3b7f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:41.469445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ac213-dfa6-46ef-8bce-4edcb45f3b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:41.500787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:41.503183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:41.503812 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8a8ac213-dfa6-46ef-8bce-4edcb45f3b7f None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:41.505203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8a8ac213-dfa6-46ef-8bce-4edcb45f3b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:41.625443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:41.651412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 979/1962] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:41.657704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ac213-dfa6-46ef-8bce-4edcb45f3b7f None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:41.659767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8a8ac213-dfa6-46ef-8bce-4edcb45f3b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:41.675601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b99269-c0e3-4bc5-b7d1-3bc1c0d3594b admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:41.692345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:41.693410 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:41.694226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:41.694613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] PATH_INFO: `/v3/projects/5197c15b6f1145989cec0f17a1832c82` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:41.696429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5197c15b6f1145989cec0f17a1832c82)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:41.700004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:41.734456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ac213-dfa6-46ef-8bce-4edcb45f3b7f None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:41.738789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b4b33-ee5a-405d-8272-a89f961a9368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:41.739432 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b4b33-ee5a-405d-8272-a89f961a9368 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:41.739761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b4b33-ee5a-405d-8272-a89f961a9368 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:41.740090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b4b33-ee5a-405d-8272-a89f961a9368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:41.759421 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b4b33-ee5a-405d-8272-a89f961a9368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:41.762506 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b4b33-ee5a-405d-8272-a89f961a9368 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:41.780441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 980/1963] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:41.790100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5197c15b6f1145989cec0f17a1832c82', 'request_id': 'req-513681c5-f284-47f0-bb6b-25bcccc3db9d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:41.811081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5197c15b6f1145989cec0f17a1832c82', 'request_id': 'req-513681c5-f284-47f0-bb6b-25bcccc3db9d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:41.825297 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5197c15b6f1145989cec0f17a1832c82', 'request_id': 'req-513681c5-f284-47f0-bb6b-25bcccc3db9d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:41.847051 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] The token cache is being invalidate because project 5197c15b6f1145989cec0f17a1832c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:41.849574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:41.851158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:41.854867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 984/1964] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:41 2022] DELETE /identity/v3/projects/5197c15b6f1145989cec0f17a1832c82 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:41.860670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b4b33-ee5a-405d-8272-a89f961a9368 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:41.879273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b4b33-ee5a-405d-8272-a89f961a9368 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:42.034711 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:42.036044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:42.036489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:42.036778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] PATH_INFO: `/v3/users/f64ce609fde5497798d5bad22b44de04` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:42.038092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f64ce609fde5497798d5bad22b44de04)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:42.040296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:42.140203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f64ce609fde5497798d5bad22b44de04'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.152357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f64ce609fde5497798d5bad22b44de04'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.153247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f64ce609fde5497798d5bad22b44de04'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.195868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f64ce609fde5497798d5bad22b44de04'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.209543 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f64ce609fde5497798d5bad22b44de04'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.237128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 981/1965] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:41 2022] DELETE /identity/v3/users/f64ce609fde5497798d5bad22b44de04 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:42.307547 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:42.309470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:42.334269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513681c5-f284-47f0-bb6b-25bcccc3db9d admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:42.418977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:42.455007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:42.455931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:42.456946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:42.457392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] PATH_INFO: `/v3/projects/cda2735fb7ea4e25aae786534e933c2c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:42.460057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda2735fb7ea4e25aae786534e933c2c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:42.466790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:42.539621 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda2735fb7ea4e25aae786534e933c2c', 'request_id': 'req-753fcd0c-c8c6-443f-a3b7-60fb706bf968'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.546256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda2735fb7ea4e25aae786534e933c2c', 'request_id': 'req-753fcd0c-c8c6-443f-a3b7-60fb706bf968'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.555213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda2735fb7ea4e25aae786534e933c2c', 'request_id': 'req-753fcd0c-c8c6-443f-a3b7-60fb706bf968'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.568000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] The token cache is being invalidate because project cda2735fb7ea4e25aae786534e933c2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:42.568410 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.569092 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.570691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 985/1966] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:42 2022] DELETE /identity/v3/projects/cda2735fb7ea4e25aae786534e933c2c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:42.574876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:42.576452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:42.590578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8e157-3225-4c7b-a6b9-7ed54b2576fd admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:42.620053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:42.682140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc5d8d-1083-4f1e-b8c7-d0397c8c3293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:42.682799 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc5d8d-1083-4f1e-b8c7-d0397c8c3293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:42.683188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc5d8d-1083-4f1e-b8c7-d0397c8c3293 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:42.683539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc5d8d-1083-4f1e-b8c7-d0397c8c3293 admin admin] PATH_INFO: `/v3/users/2bddfac2635045f0a7ea9fbcc6c5a9f2` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:42.684886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc5d8d-1083-4f1e-b8c7-d0397c8c3293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bddfac2635045f0a7ea9fbcc6c5a9f2)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:42.687075 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc5d8d-1083-4f1e-b8c7-d0397c8c3293 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:42.746001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:42.746001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:42.755448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-93bc5d8d-1083-4f1e-b8c7-d0397c8c3293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bddfac2635045f0a7ea9fbcc6c5a9f2'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.766350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fcd0c-c8c6-443f-a3b7-60fb706bf968 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:42.768551 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-93bc5d8d-1083-4f1e-b8c7-d0397c8c3293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bddfac2635045f0a7ea9fbcc6c5a9f2'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.769319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-93bc5d8d-1083-4f1e-b8c7-d0397c8c3293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bddfac2635045f0a7ea9fbcc6c5a9f2'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.777115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fd126-2a41-49f9-a911-9d55c4a7734e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:42.778211 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fd126-2a41-49f9-a911-9d55c4a7734e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:42.778563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fd126-2a41-49f9-a911-9d55c4a7734e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:42.778886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fd126-2a41-49f9-a911-9d55c4a7734e admin admin] PATH_INFO: `/v3/projects/bd2f83b76f5444caa020bbc2a77b4bd3` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:42.780103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7fd126-2a41-49f9-a911-9d55c4a7734e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd2f83b76f5444caa020bbc2a77b4bd3)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:42.782840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7fd126-2a41-49f9-a911-9d55c4a7734e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:42.785175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-93bc5d8d-1083-4f1e-b8c7-d0397c8c3293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bddfac2635045f0a7ea9fbcc6c5a9f2'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.795007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-93bc5d8d-1083-4f1e-b8c7-d0397c8c3293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bddfac2635045f0a7ea9fbcc6c5a9f2'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.801972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 982/1967] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:42 2022] DELETE /identity/v3/users/2bddfac2635045f0a7ea9fbcc6c5a9f2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:42.848968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8c7fd126-2a41-49f9-a911-9d55c4a7734e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd2f83b76f5444caa020bbc2a77b4bd3', 'request_id': 'req-8c7fd126-2a41-49f9-a911-9d55c4a7734e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.856245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8c7fd126-2a41-49f9-a911-9d55c4a7734e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd2f83b76f5444caa020bbc2a77b4bd3', 'request_id': 'req-8c7fd126-2a41-49f9-a911-9d55c4a7734e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.869661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8c7fd126-2a41-49f9-a911-9d55c4a7734e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd2f83b76f5444caa020bbc2a77b4bd3', 'request_id': 'req-8c7fd126-2a41-49f9-a911-9d55c4a7734e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.885571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8c7fd126-2a41-49f9-a911-9d55c4a7734e admin admin] The token cache is being invalidate because project bd2f83b76f5444caa020bbc2a77b4bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:42.886224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8c7fd126-2a41-49f9-a911-9d55c4a7734e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.887182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-8c7fd126-2a41-49f9-a911-9d55c4a7734e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:42.889694 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 986/1968] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:42 2022] DELETE /identity/v3/projects/bd2f83b76f5444caa020bbc2a77b4bd3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:42.898508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8b1b3-fe7f-4bd4-95cb-5a04666f4e4e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:42.899771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8b1b3-fe7f-4bd4-95cb-5a04666f4e4e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:42.902773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8b1b3-fe7f-4bd4-95cb-5a04666f4e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:42.927989 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-37e8b1b3-fe7f-4bd4-95cb-5a04666f4e4e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:42.929239 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-37e8b1b3-fe7f-4bd4-95cb-5a04666f4e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:43.015702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 983/1969] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:43.020014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682f74c-e0b4-4c8d-b5c7-a16ba43c6350 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:43.020424 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682f74c-e0b4-4c8d-b5c7-a16ba43c6350 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:43.020769 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682f74c-e0b4-4c8d-b5c7-a16ba43c6350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:43.042974 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a682f74c-e0b4-4c8d-b5c7-a16ba43c6350 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:43.044274 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a682f74c-e0b4-4c8d-b5c7-a16ba43c6350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:43.056100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 987/1970] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:43.060706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8b1b3-fe7f-4bd4-95cb-5a04666f4e4e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:43.070721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8b1b3-fe7f-4bd4-95cb-5a04666f4e4e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:43.076697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49e31e-a841-44cd-b302-58236bb66607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:43.077660 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49e31e-a841-44cd-b302-58236bb66607 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:43.077660 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49e31e-a841-44cd-b302-58236bb66607 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:43.077852 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49e31e-a841-44cd-b302-58236bb66607 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:43.078540 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49e31e-a841-44cd-b302-58236bb66607 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:43.079042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49e31e-a841-44cd-b302-58236bb66607 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:43.080862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49e31e-a841-44cd-b302-58236bb66607 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:43.095034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 984/1971] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:43.099748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682f74c-e0b4-4c8d-b5c7-a16ba43c6350 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:43.114523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682f74c-e0b4-4c8d-b5c7-a16ba43c6350 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:43.122053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075e47a-a6e9-4b32-b399-bb8afd03be41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:43.122688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075e47a-a6e9-4b32-b399-bb8afd03be41 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:43.123043 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075e47a-a6e9-4b32-b399-bb8afd03be41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:43.123331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075e47a-a6e9-4b32-b399-bb8afd03be41 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:43.124013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075e47a-a6e9-4b32-b399-bb8afd03be41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:43.125952 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075e47a-a6e9-4b32-b399-bb8afd03be41 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:43.157845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 988/1972] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:43 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:43.161955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49e31e-a841-44cd-b302-58236bb66607 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:43.172727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49e31e-a841-44cd-b302-58236bb66607 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:43.206191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075e47a-a6e9-4b32-b399-bb8afd03be41 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:43.207796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:43.208683 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8075e47a-a6e9-4b32-b399-bb8afd03be41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:43.253669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:43.257612 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8505525-db02-421f-810e-ea8a58cf1665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:43.258243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8505525-db02-421f-810e-ea8a58cf1665 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:43.258541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8505525-db02-421f-810e-ea8a58cf1665 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:43.258815 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8505525-db02-421f-810e-ea8a58cf1665 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:43.261974 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8505525-db02-421f-810e-ea8a58cf1665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:43.263073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8505525-db02-421f-810e-ea8a58cf1665 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:43.290641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075e47a-a6e9-4b32-b399-bb8afd03be41 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:43.294840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 985/1973] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:43 2022] POST /identity/v3/users => generated 284 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:43.298443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8505525-db02-421f-810e-ea8a58cf1665 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:43.300560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:43.303474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:43.304670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:43.306671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] PATH_INFO: `/v3/projects/de20524a38c44e57b2634d3f4d63e348` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:43.310873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] RBAC: Authorizing `identity:delete_project(project_id=de20524a38c44e57b2634d3f4d63e348)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:43.311488 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8505525-db02-421f-810e-ea8a58cf1665 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:43.315176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:43.320048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455db6e-c8b5-4d75-8132-c0e51ba65d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:43.320509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455db6e-c8b5-4d75-8132-c0e51ba65d59 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:43.320854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455db6e-c8b5-4d75-8132-c0e51ba65d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:43.321262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455db6e-c8b5-4d75-8132-c0e51ba65d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:43.321922 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455db6e-c8b5-4d75-8132-c0e51ba65d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:43.324547 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455db6e-c8b5-4d75-8132-c0e51ba65d59 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:43.345503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 986/1974] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:43.349357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455db6e-c8b5-4d75-8132-c0e51ba65d59 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:43.369151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455db6e-c8b5-4d75-8132-c0e51ba65d59 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:43.388823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fc724-a1f3-49de-8c75-4bfc549ebc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:43.389660 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fc724-a1f3-49de-8c75-4bfc549ebc8a admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:43.390152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fc724-a1f3-49de-8c75-4bfc549ebc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:43.390616 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fc724-a1f3-49de-8c75-4bfc549ebc8a admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:43.391562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fc724-a1f3-49de-8c75-4bfc549ebc8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:43.394081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fc724-a1f3-49de-8c75-4bfc549ebc8a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:43.406311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de20524a38c44e57b2634d3f4d63e348', 'request_id': 'req-64f28746-d4cb-4041-8cc7-c3fe0df4318d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:43.413990 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de20524a38c44e57b2634d3f4d63e348', 'request_id': 'req-64f28746-d4cb-4041-8cc7-c3fe0df4318d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:43.414542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 987/1975] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:43.416878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fc724-a1f3-49de-8c75-4bfc549ebc8a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:43.423820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de20524a38c44e57b2634d3f4d63e348', 'request_id': 'req-64f28746-d4cb-4041-8cc7-c3fe0df4318d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:43.429799 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fc724-a1f3-49de-8c75-4bfc549ebc8a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:43.443352 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] The token cache is being invalidate because project de20524a38c44e57b2634d3f4d63e348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:43.443845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:43.445466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:43.448963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 989/1976] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:43 2022] DELETE /identity/v3/projects/de20524a38c44e57b2634d3f4d63e348 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:43.453508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:43.457231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:43.523906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075b430-0f54-4b9c-9f33-e8288ad7084a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:43.524766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075b430-0f54-4b9c-9f33-e8288ad7084a admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:43.525219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075b430-0f54-4b9c-9f33-e8288ad7084a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:43.525603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075b430-0f54-4b9c-9f33-e8288ad7084a admin admin] PATH_INFO: `/v3/projects/2ee298edae7f4119af1f485e5bbc5a52/users/c740ed2d47374d139e8e0da7b1e43be4/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:43.551027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075b430-0f54-4b9c-9f33-e8288ad7084a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee298edae7f4119af1f485e5bbc5a52, user_id=c740ed2d47374d139e8e0da7b1e43be4, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:43.554205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075b430-0f54-4b9c-9f33-e8288ad7084a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:43.578579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 988/1977] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:43 2022] PUT /identity/v3/projects/2ee298edae7f4119af1f485e5bbc5a52/users/c740ed2d47374d139e8e0da7b1e43be4/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:43.591002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf86805-ae9e-4bc8-a49b-063a7c23ffb6 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:43.591312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf86805-ae9e-4bc8-a49b-063a7c23ffb6 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:43.591558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf86805-ae9e-4bc8-a49b-063a7c23ffb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:43.597920 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f28746-d4cb-4041-8cc7-c3fe0df4318d admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:43.627761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-9bf86805-ae9e-4bc8-a49b-063a7c23ffb6 None None] MFA Rules not processed for user `c740ed2d47374d139e8e0da7b1e43be4`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:43.628956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9bf86805-ae9e-4bc8-a49b-063a7c23ffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:43.702982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:43.703563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:43.703913 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:43.704203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] PATH_INFO: `/v3/users/eee90e29bad946829c5b85e530f0f5c0` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:43.705532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee90e29bad946829c5b85e530f0f5c0)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:43.708040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:43.724097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:43.744931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 989/1978] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:43.746216 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf86805-ae9e-4bc8-a49b-063a7c23ffb6 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:43.747995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9bf86805-ae9e-4bc8-a49b-063a7c23ffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:43.769248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf86805-ae9e-4bc8-a49b-063a7c23ffb6 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:43.775611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b44b3-ff85-4ee5-92ba-09a29dc0c5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:43.776757 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b44b3-ff85-4ee5-92ba-09a29dc0c5bf admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:43.777049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b44b3-ff85-4ee5-92ba-09a29dc0c5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:43.777321 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b44b3-ff85-4ee5-92ba-09a29dc0c5bf admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:43.777973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b44b3-ff85-4ee5-92ba-09a29dc0c5bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:43.779956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b44b3-ff85-4ee5-92ba-09a29dc0c5bf admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:43.789216 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:43.795719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee90e29bad946829c5b85e530f0f5c0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:43.805511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee90e29bad946829c5b85e530f0f5c0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:43.806377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee90e29bad946829c5b85e530f0f5c0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:43.808593 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee90e29bad946829c5b85e530f0f5c0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:43.824768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee90e29bad946829c5b85e530f0f5c0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:43.842316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 990/1979] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:43 2022] DELETE /identity/v3/users/eee90e29bad946829c5b85e530f0f5c0 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:43.851026 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 990/1980] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:43 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:43.852665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:43.856027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:44.007450 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3020f-d6eb-49b6-bca3-e81cdef81443 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:44.137701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce7372-74c9-4e48-9eb1-e5ce23774502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:44.138430 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce7372-74c9-4e48-9eb1-e5ce23774502 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:44.138661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce7372-74c9-4e48-9eb1-e5ce23774502 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:44.138946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce7372-74c9-4e48-9eb1-e5ce23774502 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:44.140994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce7372-74c9-4e48-9eb1-e5ce23774502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:44.141902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce7372-74c9-4e48-9eb1-e5ce23774502 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:44.173041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 991/1981] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:43 2022] POST /identity/v3/users => generated 291 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:44.175409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b44b3-ff85-4ee5-92ba-09a29dc0c5bf admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:44.194656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b44b3-ff85-4ee5-92ba-09a29dc0c5bf admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:44.207883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9aa56-7ca5-4d0d-a1f6-61f64002fa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:44.208719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9aa56-7ca5-4d0d-a1f6-61f64002fa5d admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:44.208844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9aa56-7ca5-4d0d-a1f6-61f64002fa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:44.209059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9aa56-7ca5-4d0d-a1f6-61f64002fa5d admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:44.211710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9aa56-7ca5-4d0d-a1f6-61f64002fa5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:44.214139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9aa56-7ca5-4d0d-a1f6-61f64002fa5d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:44.216640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce7372-74c9-4e48-9eb1-e5ce23774502 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:44.224019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-62ce7372-74c9-4e48-9eb1-e5ce23774502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:44.243263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce7372-74c9-4e48-9eb1-e5ce23774502 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:44.246184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 991/1982] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:44.252770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9aa56-7ca5-4d0d-a1f6-61f64002fa5d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:44.263924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:44.264596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:44.264872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:44.265144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] PATH_INFO: `/v3/users/c0d9ab94cf6846e7bde6eb86b40dd29d` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:44.266462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0d9ab94cf6846e7bde6eb86b40dd29d)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:44.267564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9aa56-7ca5-4d0d-a1f6-61f64002fa5d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:44.268958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:44.274063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c83f29-e0ab-46c1-8ee4-01098e90f103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:44.275444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c83f29-e0ab-46c1-8ee4-01098e90f103 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:44.277268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c83f29-e0ab-46c1-8ee4-01098e90f103 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:44.277268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c83f29-e0ab-46c1-8ee4-01098e90f103 admin admin] PATH_INFO: `/v3/projects/a72b3e07faab4af3b76e23e4cffaf92f/users/78fb11b2d91b456084a064367270f38b/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:44.301577 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:44.311956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c83f29-e0ab-46c1-8ee4-01098e90f103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a72b3e07faab4af3b76e23e4cffaf92f, user_id=78fb11b2d91b456084a064367270f38b, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:44.315931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c83f29-e0ab-46c1-8ee4-01098e90f103 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:44.342774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 992/1983] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:44 2022] PUT /identity/v3/projects/a72b3e07faab4af3b76e23e4cffaf92f/users/78fb11b2d91b456084a064367270f38b/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:44.343872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c83f29-e0ab-46c1-8ee4-01098e90f103 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:44.371733 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0d9ab94cf6846e7bde6eb86b40dd29d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:44.389591 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0d9ab94cf6846e7bde6eb86b40dd29d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:44.397881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0d9ab94cf6846e7bde6eb86b40dd29d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:44.400007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0d9ab94cf6846e7bde6eb86b40dd29d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:44.458692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0d9ab94cf6846e7bde6eb86b40dd29d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:44.496172 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 992/1984] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:44 2022] DELETE /identity/v3/users/c0d9ab94cf6846e7bde6eb86b40dd29d => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:44.497094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:44.498688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:44.509240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c83f29-e0ab-46c1-8ee4-01098e90f103 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:44.585496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293086e-6d14-4730-8cc0-bb776528495e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:44.586171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293086e-6d14-4730-8cc0-bb776528495e admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:44.586454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293086e-6d14-4730-8cc0-bb776528495e admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:44.586732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293086e-6d14-4730-8cc0-bb776528495e admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:44.587424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293086e-6d14-4730-8cc0-bb776528495e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:44.589685 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293086e-6d14-4730-8cc0-bb776528495e admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:44.603359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085b9e1-7472-46ca-93de-334489898782 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:44.610107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 993/1985] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:44.610404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:44.610782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:44.611065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:44.611344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] PATH_INFO: `/v3/projects/279a7184a41847228c7741786b1ec087` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:44.612308 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094955cb-b9e1-4725-b65b-965d0bf366ae None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:44.612579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=279a7184a41847228c7741786b1ec087)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:44.612769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094955cb-b9e1-4725-b65b-965d0bf366ae None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:44.613279 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094955cb-b9e1-4725-b65b-965d0bf366ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:44.619717 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:44.644786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-094955cb-b9e1-4725-b65b-965d0bf366ae None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:44.646388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-094955cb-b9e1-4725-b65b-965d0bf366ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:44.658934 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 994/1986] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:44.662443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094955cb-b9e1-4725-b65b-965d0bf366ae None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:44.663214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-094955cb-b9e1-4725-b65b-965d0bf366ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:44.677450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094955cb-b9e1-4725-b65b-965d0bf366ae None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:44.692004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '279a7184a41847228c7741786b1ec087', 'request_id': 'req-70a442f6-8ab4-4469-88f3-1fab22a34d49'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:44.701698 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '279a7184a41847228c7741786b1ec087', 'request_id': 'req-70a442f6-8ab4-4469-88f3-1fab22a34d49'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:44.716499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '279a7184a41847228c7741786b1ec087', 'request_id': 'req-70a442f6-8ab4-4469-88f3-1fab22a34d49'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:44.730936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] The token cache is being invalidate because project 279a7184a41847228c7741786b1ec087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:44.731323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:44.732292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:44.734083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 993/1987] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:44 2022] DELETE /identity/v3/projects/279a7184a41847228c7741786b1ec087 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:44.737068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:44.739612 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:44.753300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:44.770988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be5d56-aa49-4c9a-8557-94fb0f0e731c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:44.771872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be5d56-aa49-4c9a-8557-94fb0f0e731c admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:44.772425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be5d56-aa49-4c9a-8557-94fb0f0e731c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:44.772780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be5d56-aa49-4c9a-8557-94fb0f0e731c admin admin] PATH_INFO: `/v3/projects/a72b3e07faab4af3b76e23e4cffaf92f/users/78fb11b2d91b456084a064367270f38b/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:44.775753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be5d56-aa49-4c9a-8557-94fb0f0e731c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a72b3e07faab4af3b76e23e4cffaf92f, user_id=78fb11b2d91b456084a064367270f38b, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:44.779142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be5d56-aa49-4c9a-8557-94fb0f0e731c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:44.801552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 995/1988] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:44 2022] PUT /identity/v3/projects/a72b3e07faab4af3b76e23e4cffaf92f/users/78fb11b2d91b456084a064367270f38b/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:44.803129 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6cab4-66d4-46c7-a753-825f787e55d2 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:44.803514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6cab4-66d4-46c7-a753-825f787e55d2 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:44.803893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6cab4-66d4-46c7-a753-825f787e55d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:44.815073 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:44.846229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-60b6cab4-66d4-46c7-a753-825f787e55d2 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:44.848360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-60b6cab4-66d4-46c7-a753-825f787e55d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:44.874500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a442f6-8ab4-4469-88f3-1fab22a34d49 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:44.953187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e1d1d-98a7-4706-91c5-6ddea09753e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:44.953874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e1d1d-98a7-4706-91c5-6ddea09753e9 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:44.954101 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e1d1d-98a7-4706-91c5-6ddea09753e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:44.954376 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e1d1d-98a7-4706-91c5-6ddea09753e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:44.955344 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 996/1989] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:44.956506 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-711e1d1d-98a7-4706-91c5-6ddea09753e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:44.962166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1c096-c0eb-4cc2-a7b3-0a34f6f8b3f5 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:44.962750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1c096-c0eb-4cc2-a7b3-0a34f6f8b3f5 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:44.963413 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1c096-c0eb-4cc2-a7b3-0a34f6f8b3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:44.979452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e1d1d-98a7-4706-91c5-6ddea09753e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:44.981071 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e1d1d-98a7-4706-91c5-6ddea09753e9 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:44.992287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 994/1990] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:44.994060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165e56f-9e6f-4070-999e-7fb1639ab6d0 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:44.994918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165e56f-9e6f-4070-999e-7fb1639ab6d0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:44.995260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165e56f-9e6f-4070-999e-7fb1639ab6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:45.012842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3dc1c096-c0eb-4cc2-a7b3-0a34f6f8b3f5 None None] MFA Rules not processed for user `78fb11b2d91b456084a064367270f38b`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:45.015823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3dc1c096-c0eb-4cc2-a7b3-0a34f6f8b3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:45.025349 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-c165e56f-9e6f-4070-999e-7fb1639ab6d0 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:45.027017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c165e56f-9e6f-4070-999e-7fb1639ab6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:45.041419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 995/1991] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:45.042627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165e56f-9e6f-4070-999e-7fb1639ab6d0 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:45.054180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165e56f-9e6f-4070-999e-7fb1639ab6d0 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:45.061365 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ba9a0-05e3-45e2-9c7b-dd6cb4de1d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:45.062449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ba9a0-05e3-45e2-9c7b-dd6cb4de1d25 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:45.062803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ba9a0-05e3-45e2-9c7b-dd6cb4de1d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:45.063087 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ba9a0-05e3-45e2-9c7b-dd6cb4de1d25 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:45.064867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ba9a0-05e3-45e2-9c7b-dd6cb4de1d25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:45.065377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ba9a0-05e3-45e2-9c7b-dd6cb4de1d25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:45.067403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ba9a0-05e3-45e2-9c7b-dd6cb4de1d25 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:45.081078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 996/1992] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:45.085816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ba9a0-05e3-45e2-9c7b-dd6cb4de1d25 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:45.099770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ba9a0-05e3-45e2-9c7b-dd6cb4de1d25 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:45.104811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecd326-ecc6-4729-ab6a-d2c4784e37d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:45.105648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecd326-ecc6-4729-ab6a-d2c4784e37d2 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:45.105996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecd326-ecc6-4729-ab6a-d2c4784e37d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:45.106276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecd326-ecc6-4729-ab6a-d2c4784e37d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:45.107444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecd326-ecc6-4729-ab6a-d2c4784e37d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:45.109419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecd326-ecc6-4729-ab6a-d2c4784e37d2 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:45.130293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 997/1993] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:45.151529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1c096-c0eb-4cc2-a7b3-0a34f6f8b3f5 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:45.153800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3dc1c096-c0eb-4cc2-a7b3-0a34f6f8b3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:45.169020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 997/1994] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:45 2022] POST /identity/v3/projects => generated 393 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:45.170146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecd326-ecc6-4729-ab6a-d2c4784e37d2 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:45.277568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1c096-c0eb-4cc2-a7b3-0a34f6f8b3f5 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:45.282779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecd326-ecc6-4729-ab6a-d2c4784e37d2 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:45.289611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68f925-2c98-4d68-af90-ba782e5af980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:45.290836 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68f925-2c98-4d68-af90-ba782e5af980 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:45.291903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68f925-2c98-4d68-af90-ba782e5af980 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:45.292401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68f925-2c98-4d68-af90-ba782e5af980 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:45.293267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68f925-2c98-4d68-af90-ba782e5af980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:45.295574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68f925-2c98-4d68-af90-ba782e5af980 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:45.329452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 998/1995] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:45 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:45.332810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68f925-2c98-4d68-af90-ba782e5af980 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:45.339126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:45.347460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68f925-2c98-4d68-af90-ba782e5af980 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:45.355726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656e5ab-4773-4926-86d6-c1823bfc563f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:45.358085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656e5ab-4773-4926-86d6-c1823bfc563f admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:45.358616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656e5ab-4773-4926-86d6-c1823bfc563f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:45.359854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656e5ab-4773-4926-86d6-c1823bfc563f admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:45.364066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656e5ab-4773-4926-86d6-c1823bfc563f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:45.366047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656e5ab-4773-4926-86d6-c1823bfc563f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:45.392629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 999/1996] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:45.392629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656e5ab-4773-4926-86d6-c1823bfc563f admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:45.395435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e656e5ab-4773-4926-86d6-c1823bfc563f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:45.408591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bbd3f-257d-4d37-b431-7d5d6a760452 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:45.409575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bbd3f-257d-4d37-b431-7d5d6a760452 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:45.409575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bbd3f-257d-4d37-b431-7d5d6a760452 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:45.418093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656e5ab-4773-4926-86d6-c1823bfc563f admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:45.419168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bbd3f-257d-4d37-b431-7d5d6a760452 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:45.425401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:45.426104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:45.426433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:45.426594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] PATH_INFO: `/v3/projects/10d7741ed2f14136b7865fac9d3371dc` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:45.427867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d7741ed2f14136b7865fac9d3371dc)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:45.429712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:45.463796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:45.517068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d7741ed2f14136b7865fac9d3371dc', 'request_id': 'req-47731edb-0bbf-421e-a121-5c1f7b03d8bc'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:45.526654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d7741ed2f14136b7865fac9d3371dc', 'request_id': 'req-47731edb-0bbf-421e-a121-5c1f7b03d8bc'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:45.533471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bbd3f-257d-4d37-b431-7d5d6a760452 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:45.535463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bbd3f-257d-4d37-b431-7d5d6a760452 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:45.539933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d7741ed2f14136b7865fac9d3371dc', 'request_id': 'req-47731edb-0bbf-421e-a121-5c1f7b03d8bc'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:45.550844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 998/1997] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:45.551968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bbd3f-257d-4d37-b431-7d5d6a760452 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:45.556031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] The token cache is being invalidate because project 10d7741ed2f14136b7865fac9d3371dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:45.556364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:45.557360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:45.558838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1000/1998] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:45 2022] DELETE /identity/v3/projects/10d7741ed2f14136b7865fac9d3371dc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:45.563348 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:45.564127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bbd3f-257d-4d37-b431-7d5d6a760452 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:45.565002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:45.656088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63164ffe-9198-4928-b46c-8918fde30e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:45.656756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63164ffe-9198-4928-b46c-8918fde30e73 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:45.657374 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63164ffe-9198-4928-b46c-8918fde30e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:45.657374 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63164ffe-9198-4928-b46c-8918fde30e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:45.657949 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63164ffe-9198-4928-b46c-8918fde30e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:45.659573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63164ffe-9198-4928-b46c-8918fde30e73 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:45.668783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47731edb-0bbf-421e-a121-5c1f7b03d8bc admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:45.675458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:45.676293 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:45.676645 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:45.677015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] PATH_INFO: `/v3/users/abfedf7b5d9641bb8472d60fb22a44c2` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:45.678336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abfedf7b5d9641bb8472d60fb22a44c2)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:45.679519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 999/1999] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:45.680793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:45.684673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63164ffe-9198-4928-b46c-8918fde30e73 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:45.687418 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-63164ffe-9198-4928-b46c-8918fde30e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:45.703548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63164ffe-9198-4928-b46c-8918fde30e73 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:45.710152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529a120-2afb-4810-b6ac-8690f71fef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:45.710950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529a120-2afb-4810-b6ac-8690f71fef4a admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:45.712103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529a120-2afb-4810-b6ac-8690f71fef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:45.712103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529a120-2afb-4810-b6ac-8690f71fef4a admin admin] PATH_INFO: `/v3/projects/ce05833464264433a3ec27827de46437/users/164a995758244cc0a4f89f1fb52fe281/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:45.737142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529a120-2afb-4810-b6ac-8690f71fef4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce05833464264433a3ec27827de46437, user_id=164a995758244cc0a4f89f1fb52fe281, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:45.740314 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529a120-2afb-4810-b6ac-8690f71fef4a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:45.767136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1000/2000] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:45 2022] PUT /identity/v3/projects/ce05833464264433a3ec27827de46437/users/164a995758244cc0a4f89f1fb52fe281/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:45.771975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abfedf7b5d9641bb8472d60fb22a44c2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:45.781639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec54d35-7732-4996-98b7-9eb887653e1e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:45.782178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec54d35-7732-4996-98b7-9eb887653e1e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:45.782578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec54d35-7732-4996-98b7-9eb887653e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:45.794861 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abfedf7b5d9641bb8472d60fb22a44c2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:45.796125 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abfedf7b5d9641bb8472d60fb22a44c2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:45.797443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abfedf7b5d9641bb8472d60fb22a44c2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:45.827078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abfedf7b5d9641bb8472d60fb22a44c2'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:45.851210 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-fec54d35-7732-4996-98b7-9eb887653e1e None None] MFA Rules not processed for user `164a995758244cc0a4f89f1fb52fe281`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:45.852578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1001/2001] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:45 2022] DELETE /identity/v3/users/abfedf7b5d9641bb8472d60fb22a44c2 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:45.852763 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-fec54d35-7732-4996-98b7-9eb887653e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:45.922410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:46.013249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:46.015224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:46.017821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1001/2002] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:46.073493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:46.124445 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae34177-4330-49d2-b43b-3aaef98d2cb1 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:46.144528 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87fb77-56d1-43bd-bf68-2ad879ecee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:46.146619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87fb77-56d1-43bd-bf68-2ad879ecee10 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:46.146966 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87fb77-56d1-43bd-bf68-2ad879ecee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:46.147574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87fb77-56d1-43bd-bf68-2ad879ecee10 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:46.149265 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87fb77-56d1-43bd-bf68-2ad879ecee10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:46.153550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87fb77-56d1-43bd-bf68-2ad879ecee10 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:46.198585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec54d35-7732-4996-98b7-9eb887653e1e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:46.210637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1002/2003] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:46 2022] POST /identity/v3/projects => generated 393 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:46.214118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec54d35-7732-4996-98b7-9eb887653e1e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:46.312994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffba1a-0626-43af-9c44-3e55eda139ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:46.313907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffba1a-0626-43af-9c44-3e55eda139ba admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:46.314378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffba1a-0626-43af-9c44-3e55eda139ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:46.314840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffba1a-0626-43af-9c44-3e55eda139ba admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:46.316021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ffba1a-0626-43af-9c44-3e55eda139ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:46.319809 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ffba1a-0626-43af-9c44-3e55eda139ba admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:46.322151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87fb77-56d1-43bd-bf68-2ad879ecee10 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:46.329165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6c87fb77-56d1-43bd-bf68-2ad879ecee10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:46.364755 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87fb77-56d1-43bd-bf68-2ad879ecee10 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:46.374842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafe606-25de-4b75-bf7d-543b8aed2a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:46.375931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafe606-25de-4b75-bf7d-543b8aed2a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:46.376464 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafe606-25de-4b75-bf7d-543b8aed2a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:46.376939 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafe606-25de-4b75-bf7d-543b8aed2a32 admin admin] PATH_INFO: `/v3/projects/9710621d62ec484385b2666731ae6c4e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:46.380038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafe606-25de-4b75-bf7d-543b8aed2a32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9710621d62ec484385b2666731ae6c4e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:46.383950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafe606-25de-4b75-bf7d-543b8aed2a32 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:46.387880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1002/2004] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:46 2022] POST /identity/v3/users => generated 303 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:46.393775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffba1a-0626-43af-9c44-3e55eda139ba admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:46.410773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffba1a-0626-43af-9c44-3e55eda139ba admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:46.416382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77108d-b911-4093-9266-53ee892904bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:46.417515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77108d-b911-4093-9266-53ee892904bb admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:46.417792 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77108d-b911-4093-9266-53ee892904bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:46.418072 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77108d-b911-4093-9266-53ee892904bb admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:46.419094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77108d-b911-4093-9266-53ee892904bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:46.420509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77108d-b911-4093-9266-53ee892904bb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:46.445077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1003/2005] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:46.449783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77108d-b911-4093-9266-53ee892904bb admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:46.464532 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77108d-b911-4093-9266-53ee892904bb admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:46.472620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bfc55-11c5-41ef-81ae-6c20501c0263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:46.473796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bfc55-11c5-41ef-81ae-6c20501c0263 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:46.474449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bfc55-11c5-41ef-81ae-6c20501c0263 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:46.475012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bfc55-11c5-41ef-81ae-6c20501c0263 admin admin] PATH_INFO: `/v3/projects/1278ccbd9cea4496a10e13c2b014ea3c/users/5f45788f5b704eab89e8178f9ae31f68/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:46.494892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3fafe606-25de-4b75-bf7d-543b8aed2a32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9710621d62ec484385b2666731ae6c4e', 'request_id': 'req-3fafe606-25de-4b75-bf7d-543b8aed2a32'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:46.499794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bfc55-11c5-41ef-81ae-6c20501c0263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1278ccbd9cea4496a10e13c2b014ea3c, user_id=5f45788f5b704eab89e8178f9ae31f68, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:46.510404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bfc55-11c5-41ef-81ae-6c20501c0263 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:46.518539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3fafe606-25de-4b75-bf7d-543b8aed2a32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9710621d62ec484385b2666731ae6c4e', 'request_id': 'req-3fafe606-25de-4b75-bf7d-543b8aed2a32'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:46.527809 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1004/2006] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:46 2022] PUT /identity/v3/projects/1278ccbd9cea4496a10e13c2b014ea3c/users/5f45788f5b704eab89e8178f9ae31f68/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:46.528986 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3fafe606-25de-4b75-bf7d-543b8aed2a32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9710621d62ec484385b2666731ae6c4e', 'request_id': 'req-3fafe606-25de-4b75-bf7d-543b8aed2a32'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:46.532091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bfc55-11c5-41ef-81ae-6c20501c0263 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:46.552403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3fafe606-25de-4b75-bf7d-543b8aed2a32 admin admin] The token cache is being invalidate because project 9710621d62ec484385b2666731ae6c4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:46.552953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3fafe606-25de-4b75-bf7d-543b8aed2a32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:46.557276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3fafe606-25de-4b75-bf7d-543b8aed2a32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:46.559692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1003/2007] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:46 2022] DELETE /identity/v3/projects/9710621d62ec484385b2666731ae6c4e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:46.567791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ac75e-7c25-4cf8-bc32-b26bf8cba870 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:46.568067 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ac75e-7c25-4cf8-bc32-b26bf8cba870 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:46.568324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ac75e-7c25-4cf8-bc32-b26bf8cba870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:46.605085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-1d9ac75e-7c25-4cf8-bc32-b26bf8cba870 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:46.607826 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1d9ac75e-7c25-4cf8-bc32-b26bf8cba870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:46.657809 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bfc55-11c5-41ef-81ae-6c20501c0263 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:46.664615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9f036-09ab-4512-b8ca-652a0fe6d2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:46.665256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9f036-09ab-4512-b8ca-652a0fe6d2ee admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:46.665537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9f036-09ab-4512-b8ca-652a0fe6d2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:46.665933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9f036-09ab-4512-b8ca-652a0fe6d2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:46.666707 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9f036-09ab-4512-b8ca-652a0fe6d2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:46.673833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9f036-09ab-4512-b8ca-652a0fe6d2ee admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:46.691735 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1005/2008] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:46.693265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9f036-09ab-4512-b8ca-652a0fe6d2ee admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:46.697971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c6b9f036-09ab-4512-b8ca-652a0fe6d2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:46.713208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1004/2009] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:46.713527 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9f036-09ab-4512-b8ca-652a0fe6d2ee admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:46.715543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268cf33-ce40-43af-9511-65aa19d25ee2 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:46.715929 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268cf33-ce40-43af-9511-65aa19d25ee2 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:46.716268 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268cf33-ce40-43af-9511-65aa19d25ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:46.718895 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955ec53-7a4b-4e80-a2a2-c8945b55bbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:46.719579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955ec53-7a4b-4e80-a2a2-c8945b55bbac admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:46.719955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955ec53-7a4b-4e80-a2a2-c8945b55bbac admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:46.720230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955ec53-7a4b-4e80-a2a2-c8945b55bbac admin admin] PATH_INFO: `/v3/projects/1278ccbd9cea4496a10e13c2b014ea3c/users/5f45788f5b704eab89e8178f9ae31f68/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:46.728311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955ec53-7a4b-4e80-a2a2-c8945b55bbac admin admin] RBAC: Authorizing `identity:create_grant(project_id=1278ccbd9cea4496a10e13c2b014ea3c, user_id=5f45788f5b704eab89e8178f9ae31f68, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:46.733928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955ec53-7a4b-4e80-a2a2-c8945b55bbac admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:46.753339 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-1268cf33-ce40-43af-9511-65aa19d25ee2 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:46.757730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1268cf33-ce40-43af-9511-65aa19d25ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:46.761419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1006/2010] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:46 2022] PUT /identity/v3/projects/1278ccbd9cea4496a10e13c2b014ea3c/users/5f45788f5b704eab89e8178f9ae31f68/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:46.762762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3634c3-97ef-4f0f-b444-63634390eb74 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:46.763057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3634c3-97ef-4f0f-b444-63634390eb74 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:46.763334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3634c3-97ef-4f0f-b444-63634390eb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:46.793497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:46.802543 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-ba3634c3-97ef-4f0f-b444-63634390eb74 None None] MFA Rules not processed for user `5f45788f5b704eab89e8178f9ae31f68`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:46.804491 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ba3634c3-97ef-4f0f-b444-63634390eb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:46.829677 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:46.905338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1005/2011] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:46.909646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268cf33-ce40-43af-9511-65aa19d25ee2 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:46.920109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1007/2012] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:46 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:46.920489 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268cf33-ce40-43af-9511-65aa19d25ee2 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:46.925981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf9a15-fb20-4557-9893-80c6bc2d2789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:46.927108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf9a15-fb20-4557-9893-80c6bc2d2789 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:46.927711 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf9a15-fb20-4557-9893-80c6bc2d2789 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:46.928300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf9a15-fb20-4557-9893-80c6bc2d2789 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:46.929506 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf9a15-fb20-4557-9893-80c6bc2d2789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:46.929980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf9a15-fb20-4557-9893-80c6bc2d2789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:46.931230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3634c3-97ef-4f0f-b444-63634390eb74 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:46.932787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf9a15-fb20-4557-9893-80c6bc2d2789 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:46.934801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ba3634c3-97ef-4f0f-b444-63634390eb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:46.945593 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1006/2013] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:52:46.950808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf9a15-fb20-4557-9893-80c6bc2d2789 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:46.962306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf9a15-fb20-4557-9893-80c6bc2d2789 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:46.969139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08b4c5-f12f-43e4-99e1-42fa4c3e0a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:46.969924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08b4c5-f12f-43e4-99e1-42fa4c3e0a6d admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:46.970331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08b4c5-f12f-43e4-99e1-42fa4c3e0a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:46.970707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08b4c5-f12f-43e4-99e1-42fa4c3e0a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:46.971445 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08b4c5-f12f-43e4-99e1-42fa4c3e0a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:46.974321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08b4c5-f12f-43e4-99e1-42fa4c3e0a6d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:47.007975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1007/2014] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:46 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:47.011955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08b4c5-f12f-43e4-99e1-42fa4c3e0a6d admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:47.021298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3634c3-97ef-4f0f-b444-63634390eb74 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:47.079609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b06d6f-b8d9-4a7e-84de-69c30cd281a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:47.080293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b06d6f-b8d9-4a7e-84de-69c30cd281a7 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.080573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b06d6f-b8d9-4a7e-84de-69c30cd281a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.080849 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b06d6f-b8d9-4a7e-84de-69c30cd281a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.108684 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08b4c5-f12f-43e4-99e1-42fa4c3e0a6d admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:47.114635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8b212-e2c0-4dd7-81e4-d7e008443414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:47.115270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b212-e2c0-4dd7-81e4-d7e008443414 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.115555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b212-e2c0-4dd7-81e4-d7e008443414 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.115889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b212-e2c0-4dd7-81e4-d7e008443414 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.116561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8b212-e2c0-4dd7-81e4-d7e008443414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:47.118733 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8b212-e2c0-4dd7-81e4-d7e008443414 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:47.144793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1008/2015] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:47 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:47.149052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8b212-e2c0-4dd7-81e4-d7e008443414 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:47.160139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8b212-e2c0-4dd7-81e4-d7e008443414 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:47.165225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08faea-f4cf-43b9-8437-e3cbe146576e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:47.166082 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08faea-f4cf-43b9-8437-e3cbe146576e admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.166449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08faea-f4cf-43b9-8437-e3cbe146576e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.167298 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08faea-f4cf-43b9-8437-e3cbe146576e admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.168057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d08faea-f4cf-43b9-8437-e3cbe146576e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:47.169808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d08faea-f4cf-43b9-8437-e3cbe146576e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:47.176887 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b06d6f-b8d9-4a7e-84de-69c30cd281a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:47.178407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b06d6f-b8d9-4a7e-84de-69c30cd281a7 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:47.185153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1009/2016] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:47.187556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08faea-f4cf-43b9-8437-e3cbe146576e admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:47.194497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1008/2017] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:46 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:47.201432 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08faea-f4cf-43b9-8437-e3cbe146576e admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:47.207078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b557f3-3240-4927-9e16-746095a036d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:47.207727 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b557f3-3240-4927-9e16-746095a036d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.208016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b557f3-3240-4927-9e16-746095a036d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.208296 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b557f3-3240-4927-9e16-746095a036d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.208962 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b557f3-3240-4927-9e16-746095a036d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:47.210574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b557f3-3240-4927-9e16-746095a036d7 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:47.227968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1010/2018] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:47.229994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b06d6f-b8d9-4a7e-84de-69c30cd281a7 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:47.236873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:47.243555 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b06d6f-b8d9-4a7e-84de-69c30cd281a7 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:47.249268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90cebc-a544-490d-abfe-897c06ed3356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:47.249935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90cebc-a544-490d-abfe-897c06ed3356 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.250206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90cebc-a544-490d-abfe-897c06ed3356 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.250476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90cebc-a544-490d-abfe-897c06ed3356 admin admin] PATH_INFO: `/v3/projects/0d0162674808445cbe610e8e73b8bca9/users/4c5acc66ef3d48bba0b22c87b0c91f78/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.269513 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90cebc-a544-490d-abfe-897c06ed3356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0162674808445cbe610e8e73b8bca9, user_id=4c5acc66ef3d48bba0b22c87b0c91f78, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:47.272436 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90cebc-a544-490d-abfe-897c06ed3356 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:47.273753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b557f3-3240-4927-9e16-746095a036d7 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:47.287377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b557f3-3240-4927-9e16-746095a036d7 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:47.290714 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1009/2019] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:47 2022] PUT /identity/v3/projects/0d0162674808445cbe610e8e73b8bca9/users/4c5acc66ef3d48bba0b22c87b0c91f78/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:47.301769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578d33b-52e3-4309-bf05-ff70f464ee02 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.302050 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578d33b-52e3-4309-bf05-ff70f464ee02 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.302310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578d33b-52e3-4309-bf05-ff70f464ee02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.341676 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-9578d33b-52e3-4309-bf05-ff70f464ee02 None None] MFA Rules not processed for user `4c5acc66ef3d48bba0b22c87b0c91f78`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:47.343655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9578d33b-52e3-4309-bf05-ff70f464ee02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:47.374858 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967775a7-ed3e-483a-bdb4-a4b58c2408df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:47.375480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967775a7-ed3e-483a-bdb4-a4b58c2408df service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.375808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967775a7-ed3e-483a-bdb4-a4b58c2408df service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.376076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967775a7-ed3e-483a-bdb4-a4b58c2408df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.377864 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-967775a7-ed3e-483a-bdb4-a4b58c2408df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:47.403923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:47.436657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1010/2020] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:47.452264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967775a7-ed3e-483a-bdb4-a4b58c2408df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:47.453745 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967775a7-ed3e-483a-bdb4-a4b58c2408df service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:47.463606 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1011/2021] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:47.499121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578d33b-52e3-4309-bf05-ff70f464ee02 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:47.500638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9578d33b-52e3-4309-bf05-ff70f464ee02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:47.513994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578d33b-52e3-4309-bf05-ff70f464ee02 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:47.517703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f1195-21c4-4d1e-a6e5-337dc4c723cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:47.518315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f1195-21c4-4d1e-a6e5-337dc4c723cf service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.518601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f1195-21c4-4d1e-a6e5-337dc4c723cf service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.518875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f1195-21c4-4d1e-a6e5-337dc4c723cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.529564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f1195-21c4-4d1e-a6e5-337dc4c723cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:47.531034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f1195-21c4-4d1e-a6e5-337dc4c723cf service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:47.541567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1011/2022] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:47.563147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967775a7-ed3e-483a-bdb4-a4b58c2408df service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:47.642911 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967775a7-ed3e-483a-bdb4-a4b58c2408df service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:47.643960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f1195-21c4-4d1e-a6e5-337dc4c723cf service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:47.645408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-375f1195-21c4-4d1e-a6e5-337dc4c723cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:47.648320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a832bb4-5033-49f8-a694-106c8ce63e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:47.649392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a832bb4-5033-49f8-a694-106c8ce63e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.649830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a832bb4-5033-49f8-a694-106c8ce63e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.650279 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a832bb4-5033-49f8-a694-106c8ce63e91 admin admin] PATH_INFO: `/v3/users/4c5acc66ef3d48bba0b22c87b0c91f78` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.652318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a832bb4-5033-49f8-a694-106c8ce63e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c5acc66ef3d48bba0b22c87b0c91f78)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:47.654360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a832bb4-5033-49f8-a694-106c8ce63e91 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:47.665504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f1195-21c4-4d1e-a6e5-337dc4c723cf service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:47.674326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc77b5-2569-4656-a285-bde1d33a8d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:47.675268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc77b5-2569-4656-a285-bde1d33a8d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.675584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc77b5-2569-4656-a285-bde1d33a8d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.675952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc77b5-2569-4656-a285-bde1d33a8d30 admin admin] PATH_INFO: `/v3/users/c740ed2d47374d139e8e0da7b1e43be4` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.677435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc77b5-2569-4656-a285-bde1d33a8d30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c740ed2d47374d139e8e0da7b1e43be4)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:47.683612 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc77b5-2569-4656-a285-bde1d33a8d30 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:47.718051 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7a832bb4-5033-49f8-a694-106c8ce63e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c5acc66ef3d48bba0b22c87b0c91f78'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:47.737278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7a832bb4-5033-49f8-a694-106c8ce63e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c5acc66ef3d48bba0b22c87b0c91f78'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:47.738004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7a832bb4-5033-49f8-a694-106c8ce63e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c5acc66ef3d48bba0b22c87b0c91f78'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:47.738731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7a832bb4-5033-49f8-a694-106c8ce63e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c5acc66ef3d48bba0b22c87b0c91f78'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:47.750151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fadc77b5-2569-4656-a285-bde1d33a8d30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c740ed2d47374d139e8e0da7b1e43be4'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:47.761466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7a832bb4-5033-49f8-a694-106c8ce63e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c5acc66ef3d48bba0b22c87b0c91f78'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:47.763497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fadc77b5-2569-4656-a285-bde1d33a8d30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c740ed2d47374d139e8e0da7b1e43be4'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:47.765365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fadc77b5-2569-4656-a285-bde1d33a8d30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c740ed2d47374d139e8e0da7b1e43be4'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:47.780868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fadc77b5-2569-4656-a285-bde1d33a8d30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c740ed2d47374d139e8e0da7b1e43be4'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:47.781055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1012/2023] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:47 2022] DELETE /identity/v3/users/4c5acc66ef3d48bba0b22c87b0c91f78 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:47.785468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21a8d8-dfa0-4f43-b8b8-40dc8dc50e25 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.785919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21a8d8-dfa0-4f43-b8b8-40dc8dc50e25 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.786200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21a8d8-dfa0-4f43-b8b8-40dc8dc50e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.791777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-fadc77b5-2569-4656-a285-bde1d33a8d30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c740ed2d47374d139e8e0da7b1e43be4'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:47.804177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1012/2024] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:47 2022] DELETE /identity/v3/users/c740ed2d47374d139e8e0da7b1e43be4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:47.808456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524810ce-9c83-4962-8b97-c3a52cce41fe None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.808630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524810ce-9c83-4962-8b97-c3a52cce41fe None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.808898 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524810ce-9c83-4962-8b97-c3a52cce41fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.813678 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-1f21a8d8-dfa0-4f43-b8b8-40dc8dc50e25 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:47.814621 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1f21a8d8-dfa0-4f43-b8b8-40dc8dc50e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:47.830860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-524810ce-9c83-4962-8b97-c3a52cce41fe None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:47.832231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-524810ce-9c83-4962-8b97-c3a52cce41fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:47.857973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:47.865757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:47.897192 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1013/2025] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:47.904505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21a8d8-dfa0-4f43-b8b8-40dc8dc50e25 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:47.906800 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1f21a8d8-dfa0-4f43-b8b8-40dc8dc50e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:47.918235 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1013/2026] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:47.925287 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524810ce-9c83-4962-8b97-c3a52cce41fe None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:47.926703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-524810ce-9c83-4962-8b97-c3a52cce41fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:47.978784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21a8d8-dfa0-4f43-b8b8-40dc8dc50e25 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:47.982751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f8118-ef95-4e89-871e-a1ded866d6c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:47.983563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f8118-ef95-4e89-871e-a1ded866d6c4 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:47.983898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f8118-ef95-4e89-871e-a1ded866d6c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:47.984160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f8118-ef95-4e89-871e-a1ded866d6c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:47.993896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f8118-ef95-4e89-871e-a1ded866d6c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:47.995918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f8118-ef95-4e89-871e-a1ded866d6c4 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:48.006002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1014/2027] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:48.006215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524810ce-9c83-4962-8b97-c3a52cce41fe None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:48.010834 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719a399-0730-4d45-ba71-004da91bfebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:48.011453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719a399-0730-4d45-ba71-004da91bfebc service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:48.011756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719a399-0730-4d45-ba71-004da91bfebc service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:48.012033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719a399-0730-4d45-ba71-004da91bfebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:48.021842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719a399-0730-4d45-ba71-004da91bfebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:48.023408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719a399-0730-4d45-ba71-004da91bfebc service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:48.034006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1014/2028] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:48.393185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f8118-ef95-4e89-871e-a1ded866d6c4 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:48.393185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-579f8118-ef95-4e89-871e-a1ded866d6c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:48.407039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f8118-ef95-4e89-871e-a1ded866d6c4 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:48.413133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae30d3-026b-41db-924f-1bbca51ea661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:48.413765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae30d3-026b-41db-924f-1bbca51ea661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:48.414042 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae30d3-026b-41db-924f-1bbca51ea661 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:48.414356 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae30d3-026b-41db-924f-1bbca51ea661 admin admin] PATH_INFO: `/v3/projects/0d0162674808445cbe610e8e73b8bca9` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:48.415662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae30d3-026b-41db-924f-1bbca51ea661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d0162674808445cbe610e8e73b8bca9)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:48.418159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae30d3-026b-41db-924f-1bbca51ea661 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:48.442249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719a399-0730-4d45-ba71-004da91bfebc service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:48.444035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6719a399-0730-4d45-ba71-004da91bfebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:48.468664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719a399-0730-4d45-ba71-004da91bfebc service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:48.484820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-dfae30d3-026b-41db-924f-1bbca51ea661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d0162674808445cbe610e8e73b8bca9', 'request_id': 'req-dfae30d3-026b-41db-924f-1bbca51ea661'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.490827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-dfae30d3-026b-41db-924f-1bbca51ea661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d0162674808445cbe610e8e73b8bca9', 'request_id': 'req-dfae30d3-026b-41db-924f-1bbca51ea661'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.501435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-dfae30d3-026b-41db-924f-1bbca51ea661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d0162674808445cbe610e8e73b8bca9', 'request_id': 'req-dfae30d3-026b-41db-924f-1bbca51ea661'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.515264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-dfae30d3-026b-41db-924f-1bbca51ea661 admin admin] The token cache is being invalidate because project 0d0162674808445cbe610e8e73b8bca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:48.515558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-dfae30d3-026b-41db-924f-1bbca51ea661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.516372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-dfae30d3-026b-41db-924f-1bbca51ea661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.518091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1015/2029] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:48 2022] DELETE /identity/v3/projects/0d0162674808445cbe610e8e73b8bca9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:48.522585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93ba2a-d823-49d1-b3ac-89b7fad41cb9 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:48.522877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93ba2a-d823-49d1-b3ac-89b7fad41cb9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:48.523146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93ba2a-d823-49d1-b3ac-89b7fad41cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:48.536286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:48.536929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:48.537227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:48.537502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] PATH_INFO: `/v3/projects/2ee298edae7f4119af1f485e5bbc5a52` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:48.538656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee298edae7f4119af1f485e5bbc5a52)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:48.540615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:48.545061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-bf93ba2a-d823-49d1-b3ac-89b7fad41cb9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:48.546328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bf93ba2a-d823-49d1-b3ac-89b7fad41cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:48.558212 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1016/2030] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:48.566922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd549c1-fcd9-4b8b-8ff6-55db5fcccd48 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:48.567198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd549c1-fcd9-4b8b-8ff6-55db5fcccd48 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:48.567464 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd549c1-fcd9-4b8b-8ff6-55db5fcccd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:48.589231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-9bd549c1-fcd9-4b8b-8ff6-55db5fcccd48 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:48.590466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9bd549c1-fcd9-4b8b-8ff6-55db5fcccd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:48.594682 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee298edae7f4119af1f485e5bbc5a52', 'request_id': 'req-a73ddc0d-f40d-4d96-ba30-28ae38772726'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.600787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee298edae7f4119af1f485e5bbc5a52', 'request_id': 'req-a73ddc0d-f40d-4d96-ba30-28ae38772726'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.609655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee298edae7f4119af1f485e5bbc5a52', 'request_id': 'req-a73ddc0d-f40d-4d96-ba30-28ae38772726'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.622544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] The token cache is being invalidate because project 2ee298edae7f4119af1f485e5bbc5a52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:48.622838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.623540 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.625122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1015/2031] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:48 2022] DELETE /identity/v3/projects/2ee298edae7f4119af1f485e5bbc5a52 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:48.629071 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:48.630411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:48.636037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:48.657664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1017/2032] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:48.661917 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd549c1-fcd9-4b8b-8ff6-55db5fcccd48 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:48.666259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:48.672517 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd549c1-fcd9-4b8b-8ff6-55db5fcccd48 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:48.679187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb21838-90c8-426b-b3a2-4e2f35732ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:48.680032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb21838-90c8-426b-b3a2-4e2f35732ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:48.680482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb21838-90c8-426b-b3a2-4e2f35732ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:48.680959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb21838-90c8-426b-b3a2-4e2f35732ca5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:48.683764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb21838-90c8-426b-b3a2-4e2f35732ca5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:48.683764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb21838-90c8-426b-b3a2-4e2f35732ca5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:48.683998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb21838-90c8-426b-b3a2-4e2f35732ca5 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:48.696208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1018/2033] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:52:48.700442 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb21838-90c8-426b-b3a2-4e2f35732ca5 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:48.708283 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ddc0d-f40d-4d96-ba30-28ae38772726 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:48.711238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb21838-90c8-426b-b3a2-4e2f35732ca5 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:48.713626 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:48.714310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:48.714617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:48.714888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] PATH_INFO: `/v3/users/78fb11b2d91b456084a064367270f38b` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:48.716091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=78fb11b2d91b456084a064367270f38b)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:48.717882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a1cf6-d6d1-435f-b527-48d1e4cbfe42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:48.718080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:48.718512 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a1cf6-d6d1-435f-b527-48d1e4cbfe42 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:48.718909 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a1cf6-d6d1-435f-b527-48d1e4cbfe42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:48.719163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a1cf6-d6d1-435f-b527-48d1e4cbfe42 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:48.719905 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a1cf6-d6d1-435f-b527-48d1e4cbfe42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:48.721721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a1cf6-d6d1-435f-b527-48d1e4cbfe42 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:48.755019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1019/2034] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:48 2022] POST /identity/v3/projects => generated 359 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:48.759590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a1cf6-d6d1-435f-b527-48d1e4cbfe42 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:48.770548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78fb11b2d91b456084a064367270f38b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.778548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78fb11b2d91b456084a064367270f38b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.779292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78fb11b2d91b456084a064367270f38b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.795499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78fb11b2d91b456084a064367270f38b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.804291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78fb11b2d91b456084a064367270f38b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:48.810873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1016/2035] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:48 2022] DELETE /identity/v3/users/78fb11b2d91b456084a064367270f38b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:48.832345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a1cf6-d6d1-435f-b527-48d1e4cbfe42 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:48.902492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da5b26-da2d-4277-bd1d-9d075db54a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:48.902797 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da5b26-da2d-4277-bd1d-9d075db54a70 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:48.903135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da5b26-da2d-4277-bd1d-9d075db54a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:48.903410 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da5b26-da2d-4277-bd1d-9d075db54a70 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:48.904086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da5b26-da2d-4277-bd1d-9d075db54a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:48.906231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da5b26-da2d-4277-bd1d-9d075db54a70 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:48.942713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1020/2036] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:48 2022] POST /identity/v3/users => generated 280 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:48.943980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:48.945496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:48.960394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356172a-6d69-4c43-8f7d-84c15d91bfdb admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:48.968645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9480b-3b93-4561-9a25-02c251720f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:48.969438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9480b-3b93-4561-9a25-02c251720f00 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:48.969772 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9480b-3b93-4561-9a25-02c251720f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:48.970154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9480b-3b93-4561-9a25-02c251720f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:48.971760 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9480b-3b93-4561-9a25-02c251720f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:48.973937 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9480b-3b93-4561-9a25-02c251720f00 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:48.995050 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1017/2037] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:48.999269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da5b26-da2d-4277-bd1d-9d075db54a70 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:49.014031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da5b26-da2d-4277-bd1d-9d075db54a70 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:49.019030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a679b-2191-4de0-b57e-ebe798fd318b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:49.019721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a679b-2191-4de0-b57e-ebe798fd318b admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.020018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a679b-2191-4de0-b57e-ebe798fd318b admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.020288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a679b-2191-4de0-b57e-ebe798fd318b admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.020974 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a679b-2191-4de0-b57e-ebe798fd318b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:49.022719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a679b-2191-4de0-b57e-ebe798fd318b admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:49.035539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1021/2038] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:49.039527 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9480b-3b93-4561-9a25-02c251720f00 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:49.049690 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9480b-3b93-4561-9a25-02c251720f00 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:49.054524 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd5f8e-642e-479a-a5bf-c23a7c228bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:49.055196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd5f8e-642e-479a-a5bf-c23a7c228bc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.055492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd5f8e-642e-479a-a5bf-c23a7c228bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.055807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd5f8e-642e-479a-a5bf-c23a7c228bc6 admin admin] PATH_INFO: `/v3/projects/f9bc1e9272be4386b8f865b783dba896/users/e943fb3a4124432aa210ad1a34903a40/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.076886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd5f8e-642e-479a-a5bf-c23a7c228bc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9bc1e9272be4386b8f865b783dba896, user_id=e943fb3a4124432aa210ad1a34903a40, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:49.079803 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd5f8e-642e-479a-a5bf-c23a7c228bc6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:49.095569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1018/2039] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:49 2022] PUT /identity/v3/projects/f9bc1e9272be4386b8f865b783dba896/users/e943fb3a4124432aa210ad1a34903a40/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:49.107426 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840eb01-d9b5-404f-80c0-42f63f2bbe45 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.108211 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840eb01-d9b5-404f-80c0-42f63f2bbe45 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.108558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840eb01-d9b5-404f-80c0-42f63f2bbe45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.140138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd5f8e-642e-479a-a5bf-c23a7c228bc6 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:49.142036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-79fd5f8e-642e-479a-a5bf-c23a7c228bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:49.144328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-c840eb01-d9b5-404f-80c0-42f63f2bbe45 None None] MFA Rules not processed for user `e943fb3a4124432aa210ad1a34903a40`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:49.145644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c840eb01-d9b5-404f-80c0-42f63f2bbe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:49.195517 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:49.220476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:49.233859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1022/2040] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:49.240399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd5f8e-642e-479a-a5bf-c23a7c228bc6 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:49.246897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:49.247581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.247996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.248302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4 admin admin] PATH_INFO: `/v3/projects/a72b3e07faab4af3b76e23e4cffaf92f` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.249506 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a72b3e07faab4af3b76e23e4cffaf92f)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:49.251634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:49.271335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840eb01-d9b5-404f-80c0-42f63f2bbe45 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:49.273041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c840eb01-d9b5-404f-80c0-42f63f2bbe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:49.303238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a72b3e07faab4af3b76e23e4cffaf92f', 'request_id': 'req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:49.309315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a72b3e07faab4af3b76e23e4cffaf92f', 'request_id': 'req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:49.317340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a72b3e07faab4af3b76e23e4cffaf92f', 'request_id': 'req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:49.329553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4 admin admin] The token cache is being invalidate because project a72b3e07faab4af3b76e23e4cffaf92f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:49.329846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:49.330513 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7962b05f-f12f-40a2-8dc0-9cd2aa37c6e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:49.332265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1019/2041] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:49 2022] DELETE /identity/v3/projects/a72b3e07faab4af3b76e23e4cffaf92f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:49.337030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dadd7-08b7-4a01-a7ca-8e2188209d48 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.337185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840eb01-d9b5-404f-80c0-42f63f2bbe45 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:49.337377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dadd7-08b7-4a01-a7ca-8e2188209d48 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.337592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dadd7-08b7-4a01-a7ca-8e2188209d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.358446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-002dadd7-08b7-4a01-a7ca-8e2188209d48 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:49.359697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-002dadd7-08b7-4a01-a7ca-8e2188209d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:49.389534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d857830-9a8c-41fa-bdb7-7df95b92998f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:49.390321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d857830-9a8c-41fa-bdb7-7df95b92998f service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.390597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d857830-9a8c-41fa-bdb7-7df95b92998f service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.390938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d857830-9a8c-41fa-bdb7-7df95b92998f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.392948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7d857830-9a8c-41fa-bdb7-7df95b92998f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:49.428487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1020/2042] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:49.430126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9000-2756-4536-893b-7eeb71891a97 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.430415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9000-2756-4536-893b-7eeb71891a97 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.430694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9000-2756-4536-893b-7eeb71891a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.452559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-779f9000-2756-4536-893b-7eeb71891a97 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:49.454189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-779f9000-2756-4536-893b-7eeb71891a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:49.462351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1021/2043] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:49.466524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d857830-9a8c-41fa-bdb7-7df95b92998f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:49.467009 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f9000-2756-4536-893b-7eeb71891a97 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:49.468032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d857830-9a8c-41fa-bdb7-7df95b92998f service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:49.477444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f9000-2756-4536-893b-7eeb71891a97 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:49.477985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1023/2044] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:49.482068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68788cac-09e4-409e-8615-22fefebef4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:49.482672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68788cac-09e4-409e-8615-22fefebef4cc admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.482941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68788cac-09e4-409e-8615-22fefebef4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.483216 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68788cac-09e4-409e-8615-22fefebef4cc admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.483900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68788cac-09e4-409e-8615-22fefebef4cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:49.484205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68788cac-09e4-409e-8615-22fefebef4cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:49.488604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68788cac-09e4-409e-8615-22fefebef4cc admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:49.493099 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d857830-9a8c-41fa-bdb7-7df95b92998f service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:49.495078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7d857830-9a8c-41fa-bdb7-7df95b92998f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:49.505647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1022/2045] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:52:49.506857 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68788cac-09e4-409e-8615-22fefebef4cc admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:49.509407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d857830-9a8c-41fa-bdb7-7df95b92998f service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:49.514665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac0744-882e-4ce8-926e-489979ff2bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:49.515347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac0744-882e-4ce8-926e-489979ff2bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.515626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac0744-882e-4ce8-926e-489979ff2bab admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.515933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac0744-882e-4ce8-926e-489979ff2bab admin admin] PATH_INFO: `/v3/users/e943fb3a4124432aa210ad1a34903a40` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.517094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac0744-882e-4ce8-926e-489979ff2bab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e943fb3a4124432aa210ad1a34903a40)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:49.517632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68788cac-09e4-409e-8615-22fefebef4cc admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:49.519027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac0744-882e-4ce8-926e-489979ff2bab admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:49.522783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604febb-e241-4dfc-82c5-036bc8416128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:49.523383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604febb-e241-4dfc-82c5-036bc8416128 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.523664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604febb-e241-4dfc-82c5-036bc8416128 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.523979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604febb-e241-4dfc-82c5-036bc8416128 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.524768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604febb-e241-4dfc-82c5-036bc8416128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:49.526862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604febb-e241-4dfc-82c5-036bc8416128 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:49.569290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1023/2046] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:49 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:49.569486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-31ac0744-882e-4ce8-926e-489979ff2bab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e943fb3a4124432aa210ad1a34903a40'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:49.569486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:49.570746 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604febb-e241-4dfc-82c5-036bc8416128 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:49.578106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-31ac0744-882e-4ce8-926e-489979ff2bab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e943fb3a4124432aa210ad1a34903a40'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:49.578953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-31ac0744-882e-4ce8-926e-489979ff2bab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e943fb3a4124432aa210ad1a34903a40'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:49.579939 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-31ac0744-882e-4ce8-926e-489979ff2bab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e943fb3a4124432aa210ad1a34903a40'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:49.592146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-31ac0744-882e-4ce8-926e-489979ff2bab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e943fb3a4124432aa210ad1a34903a40'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:49.606485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1024/2047] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:49 2022] DELETE /identity/v3/users/e943fb3a4124432aa210ad1a34903a40 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:49.611881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035e8b-24e2-40fb-8485-733e543b26b5 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.612175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035e8b-24e2-40fb-8485-733e543b26b5 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.612455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035e8b-24e2-40fb-8485-733e543b26b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.614864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:49.636531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-94035e8b-24e2-40fb-8485-733e543b26b5 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:49.637766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-94035e8b-24e2-40fb-8485-733e543b26b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:49.646328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604febb-e241-4dfc-82c5-036bc8416128 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:49.700991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1025/2048] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:49.710695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94035e8b-24e2-40fb-8485-733e543b26b5 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:49.710864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a43ca-51e6-4c93-a7ec-7a5431e16eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:49.711367 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a43ca-51e6-4c93-a7ec-7a5431e16eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.711711 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a43ca-51e6-4c93-a7ec-7a5431e16eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.712019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a43ca-51e6-4c93-a7ec-7a5431e16eb2 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.712182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-94035e8b-24e2-40fb-8485-733e543b26b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:49.712663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a43ca-51e6-4c93-a7ec-7a5431e16eb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:49.714594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a43ca-51e6-4c93-a7ec-7a5431e16eb2 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:49.741906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1024/2049] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:49 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:49.743830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a43ca-51e6-4c93-a7ec-7a5431e16eb2 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:49.746486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-288a43ca-51e6-4c93-a7ec-7a5431e16eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:49.758795 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a43ca-51e6-4c93-a7ec-7a5431e16eb2 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:49.763534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930b495-be25-4698-856a-5d7204b983ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:49.764177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930b495-be25-4698-856a-5d7204b983ac admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.764456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930b495-be25-4698-856a-5d7204b983ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.764735 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930b495-be25-4698-856a-5d7204b983ac admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.765379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930b495-be25-4698-856a-5d7204b983ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:49.766992 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930b495-be25-4698-856a-5d7204b983ac admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:49.777411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1025/2050] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:49.781479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930b495-be25-4698-856a-5d7204b983ac admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:49.782242 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94035e8b-24e2-40fb-8485-733e543b26b5 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:49.786054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6110f-f635-48a5-96d8-1fa29c6ce9d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:49.786671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6110f-f635-48a5-96d8-1fa29c6ce9d5 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.786952 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6110f-f635-48a5-96d8-1fa29c6ce9d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.787220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6110f-f635-48a5-96d8-1fa29c6ce9d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.792422 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930b495-be25-4698-856a-5d7204b983ac admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:49.797323 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3c34b-6a90-4618-80ad-8b65979b7bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:49.797936 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3c34b-6a90-4618-80ad-8b65979b7bbd admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.798214 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3c34b-6a90-4618-80ad-8b65979b7bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.798487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3c34b-6a90-4618-80ad-8b65979b7bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.798896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6110f-f635-48a5-96d8-1fa29c6ce9d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:49.799124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3c34b-6a90-4618-80ad-8b65979b7bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:49.800400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6110f-f635-48a5-96d8-1fa29c6ce9d5 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:49.800844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3c34b-6a90-4618-80ad-8b65979b7bbd admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:49.813602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1026/2051] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:49.814380 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1026/2052] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:49.817634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3c34b-6a90-4618-80ad-8b65979b7bbd admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:49.827727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3c34b-6a90-4618-80ad-8b65979b7bbd admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:49.832513 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294ac35-ebd6-4769-91b2-0e3cf70b0768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:49.833179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294ac35-ebd6-4769-91b2-0e3cf70b0768 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.833458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294ac35-ebd6-4769-91b2-0e3cf70b0768 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.833733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294ac35-ebd6-4769-91b2-0e3cf70b0768 admin admin] PATH_INFO: `/v3/projects/c99341e13d5143588d11576bc20b5209/users/42bd91b43e65425ab1b37a13a65cb860/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.850440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294ac35-ebd6-4769-91b2-0e3cf70b0768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99341e13d5143588d11576bc20b5209, user_id=42bd91b43e65425ab1b37a13a65cb860, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:49.853168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294ac35-ebd6-4769-91b2-0e3cf70b0768 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:49.873934 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1027/2053] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:49 2022] PUT /identity/v3/projects/c99341e13d5143588d11576bc20b5209/users/42bd91b43e65425ab1b37a13a65cb860/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:49.886950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198c859-afb7-46b8-ae35-41fa96a0f47a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:49.886950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198c859-afb7-46b8-ae35-41fa96a0f47a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:49.887420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198c859-afb7-46b8-ae35-41fa96a0f47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:49.933420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-b198c859-afb7-46b8-ae35-41fa96a0f47a None None] MFA Rules not processed for user `42bd91b43e65425ab1b37a13a65cb860`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:49.934955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b198c859-afb7-46b8-ae35-41fa96a0f47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:50.037272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:50.049058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1027/2054] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:50.052602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294ac35-ebd6-4769-91b2-0e3cf70b0768 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.138876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294ac35-ebd6-4769-91b2-0e3cf70b0768 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:50.143930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebe942-648e-446d-bff0-a6ad40b880cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:50.145257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebe942-648e-446d-bff0-a6ad40b880cc admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.145557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebe942-648e-446d-bff0-a6ad40b880cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.145831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebe942-648e-446d-bff0-a6ad40b880cc admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.146476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebe942-648e-446d-bff0-a6ad40b880cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:50.148466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebe942-648e-446d-bff0-a6ad40b880cc admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:50.177066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198c859-afb7-46b8-ae35-41fa96a0f47a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.178976 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b198c859-afb7-46b8-ae35-41fa96a0f47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:50.193822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1028/2055] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:50 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:50.199820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebe942-648e-446d-bff0-a6ad40b880cc admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.208427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198c859-afb7-46b8-ae35-41fa96a0f47a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:50.217555 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:50.284236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebe942-648e-446d-bff0-a6ad40b880cc admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:50.296023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d8e15-a1d5-4162-902d-09723fcdd538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:50.296300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:50.296552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d8e15-a1d5-4162-902d-09723fcdd538 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.296903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d8e15-a1d5-4162-902d-09723fcdd538 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.297059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d8e15-a1d5-4162-902d-09723fcdd538 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.297697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d8e15-a1d5-4162-902d-09723fcdd538 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:50.297808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.298225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.298612 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622 admin admin] PATH_INFO: `/v3/projects/f9bc1e9272be4386b8f865b783dba896` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.300266 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9bc1e9272be4386b8f865b783dba896)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:50.303583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:50.305837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d8e15-a1d5-4162-902d-09723fcdd538 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:50.338994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1029/2056] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:50 2022] POST /identity/v3/users => generated 299 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:50.338994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d8e15-a1d5-4162-902d-09723fcdd538 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.371615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9bc1e9272be4386b8f865b783dba896', 'request_id': 'req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:50.372497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d8e15-a1d5-4162-902d-09723fcdd538 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:50.379109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9bc1e9272be4386b8f865b783dba896', 'request_id': 'req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:50.392125 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9bc1e9272be4386b8f865b783dba896', 'request_id': 'req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:50.408739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622 admin admin] The token cache is being invalidate because project f9bc1e9272be4386b8f865b783dba896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:50.409303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:50.411177 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7b729d0b-c3e0-4b5b-94e1-834eb3a41622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:50.414643 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1028/2057] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:50 2022] DELETE /identity/v3/projects/f9bc1e9272be4386b8f865b783dba896 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:50.424328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136e718-0d36-40c4-b83a-dc1a53fd2421 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.424986 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136e718-0d36-40c4-b83a-dc1a53fd2421 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.425728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136e718-0d36-40c4-b83a-dc1a53fd2421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.457483 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee22632-fe19-4ad9-abb2-638c1cb7ca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:50.458773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee22632-fe19-4ad9-abb2-638c1cb7ca40 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.459290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee22632-fe19-4ad9-abb2-638c1cb7ca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.459915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee22632-fe19-4ad9-abb2-638c1cb7ca40 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.461210 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee22632-fe19-4ad9-abb2-638c1cb7ca40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:50.466013 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee22632-fe19-4ad9-abb2-638c1cb7ca40 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:50.473419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-c136e718-0d36-40c4-b83a-dc1a53fd2421 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:50.474996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c136e718-0d36-40c4-b83a-dc1a53fd2421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:50.492286 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1029/2058] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:50.492375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1030/2059] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:50.493706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136e718-0d36-40c4-b83a-dc1a53fd2421 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.494997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632bb8a-38bb-4266-acc6-d2527e7a7991 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.495121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c136e718-0d36-40c4-b83a-dc1a53fd2421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:50.495382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632bb8a-38bb-4266-acc6-d2527e7a7991 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.495837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632bb8a-38bb-4266-acc6-d2527e7a7991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.510638 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136e718-0d36-40c4-b83a-dc1a53fd2421 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:50.516537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fe7eb-3d50-48de-a8a9-c07997a366e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:50.517636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fe7eb-3d50-48de-a8a9-c07997a366e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.518116 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fe7eb-3d50-48de-a8a9-c07997a366e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.518580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fe7eb-3d50-48de-a8a9-c07997a366e5 admin admin] PATH_INFO: `/v3/projects/8d7b732ee777481c82f2b47e075d6390/users/fb81fdf05e8f44b4bd6f443ec6f0789b/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.522910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-7632bb8a-38bb-4266-acc6-d2527e7a7991 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:50.524957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7632bb8a-38bb-4266-acc6-d2527e7a7991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:50.536107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fe7eb-3d50-48de-a8a9-c07997a366e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7b732ee777481c82f2b47e075d6390, user_id=fb81fdf05e8f44b4bd6f443ec6f0789b, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:50.538716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1031/2060] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:50.538888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fe7eb-3d50-48de-a8a9-c07997a366e5 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:50.551826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632bb8a-38bb-4266-acc6-d2527e7a7991 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.567928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1030/2061] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:50 2022] PUT /identity/v3/projects/8d7b732ee777481c82f2b47e075d6390/users/fb81fdf05e8f44b4bd6f443ec6f0789b/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:50.568313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632bb8a-38bb-4266-acc6-d2527e7a7991 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:50.569348 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fe7eb-3d50-48de-a8a9-c07997a366e5 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.642140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0405b0f-f5ee-4a58-99e2-8a0b6e6dd422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:50.642837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0405b0f-f5ee-4a58-99e2-8a0b6e6dd422 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.643365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0405b0f-f5ee-4a58-99e2-8a0b6e6dd422 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.643638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0405b0f-f5ee-4a58-99e2-8a0b6e6dd422 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.644553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0405b0f-f5ee-4a58-99e2-8a0b6e6dd422 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:50.644918 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0405b0f-f5ee-4a58-99e2-8a0b6e6dd422 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:50.646822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0405b0f-f5ee-4a58-99e2-8a0b6e6dd422 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:50.659230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:50.660864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1032/2062] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:50.665500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fe7eb-3d50-48de-a8a9-c07997a366e5 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:50.666637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0405b0f-f5ee-4a58-99e2-8a0b6e6dd422 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.671895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00560d-8d2f-4e2c-8c7f-3023ff43bee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:50.672778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00560d-8d2f-4e2c-8c7f-3023ff43bee8 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.672922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00560d-8d2f-4e2c-8c7f-3023ff43bee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.673173 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00560d-8d2f-4e2c-8c7f-3023ff43bee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.673897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00560d-8d2f-4e2c-8c7f-3023ff43bee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:50.675655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00560d-8d2f-4e2c-8c7f-3023ff43bee8 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:50.682981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0405b0f-f5ee-4a58-99e2-8a0b6e6dd422 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:50.689982 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e7807-46c3-4f72-bb49-11756685be1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:50.690741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e7807-46c3-4f72-bb49-11756685be1f admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.691007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e7807-46c3-4f72-bb49-11756685be1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.691490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e7807-46c3-4f72-bb49-11756685be1f admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.691825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1031/2063] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:50.692277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e7807-46c3-4f72-bb49-11756685be1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:50.694233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e7807-46c3-4f72-bb49-11756685be1f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:50.696029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00560d-8d2f-4e2c-8c7f-3023ff43bee8 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.708152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00560d-8d2f-4e2c-8c7f-3023ff43bee8 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:50.713965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6c65b-42ea-455a-8885-0f2d00c39b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:50.714824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6c65b-42ea-455a-8885-0f2d00c39b5e admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.715207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6c65b-42ea-455a-8885-0f2d00c39b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.715814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6c65b-42ea-455a-8885-0f2d00c39b5e admin admin] PATH_INFO: `/v3/projects/8d7b732ee777481c82f2b47e075d6390/users/fb81fdf05e8f44b4bd6f443ec6f0789b/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.720973 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6c65b-42ea-455a-8885-0f2d00c39b5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7b732ee777481c82f2b47e075d6390, user_id=fb81fdf05e8f44b4bd6f443ec6f0789b, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:50.724740 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6c65b-42ea-455a-8885-0f2d00c39b5e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:50.742716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1033/2064] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:50 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:50.743877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e7807-46c3-4f72-bb49-11756685be1f admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.746203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1032/2065] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:50 2022] PUT /identity/v3/projects/8d7b732ee777481c82f2b47e075d6390/users/fb81fdf05e8f44b4bd6f443ec6f0789b/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:50.751105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:50.759064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8edcc-e808-4862-98c7-4c3ebacd1c98 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.759439 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8edcc-e808-4862-98c7-4c3ebacd1c98 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.760383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8edcc-e808-4862-98c7-4c3ebacd1c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.802384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-7bb8edcc-e808-4862-98c7-4c3ebacd1c98 None None] MFA Rules not processed for user `fb81fdf05e8f44b4bd6f443ec6f0789b`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:50.803841 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7bb8edcc-e808-4862-98c7-4c3ebacd1c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:50.840171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e7807-46c3-4f72-bb49-11756685be1f admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:50.846615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd373c1-b656-4e32-8263-6e4f637c8eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:50.847979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd373c1-b656-4e32-8263-6e4f637c8eba admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.848376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd373c1-b656-4e32-8263-6e4f637c8eba admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.848749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd373c1-b656-4e32-8263-6e4f637c8eba admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.849619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd373c1-b656-4e32-8263-6e4f637c8eba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:50.851804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd373c1-b656-4e32-8263-6e4f637c8eba admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:50.881097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1034/2066] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:50 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:50.882766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd373c1-b656-4e32-8263-6e4f637c8eba admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.903416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd373c1-b656-4e32-8263-6e4f637c8eba admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:50.909639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36193047-720d-4984-879f-8a3bb21ee355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:50.910379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36193047-720d-4984-879f-8a3bb21ee355 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.910671 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36193047-720d-4984-879f-8a3bb21ee355 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.910938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36193047-720d-4984-879f-8a3bb21ee355 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.911834 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36193047-720d-4984-879f-8a3bb21ee355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:50.913436 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36193047-720d-4984-879f-8a3bb21ee355 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:50.929037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1035/2067] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:50.930328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36193047-720d-4984-879f-8a3bb21ee355 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.935982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1033/2068] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:50.946040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36193047-720d-4984-879f-8a3bb21ee355 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:50.952002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5137d8-f1af-44f8-9288-ec13fcde8874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:50.952670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5137d8-f1af-44f8-9288-ec13fcde8874 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:50.952997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5137d8-f1af-44f8-9288-ec13fcde8874 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:50.953305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5137d8-f1af-44f8-9288-ec13fcde8874 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:50.954371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5137d8-f1af-44f8-9288-ec13fcde8874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:50.956875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5137d8-f1af-44f8-9288-ec13fcde8874 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:50.974271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1036/2069] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:50.978618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8edcc-e808-4862-98c7-4c3ebacd1c98 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.988804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5137d8-f1af-44f8-9288-ec13fcde8874 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:50.990572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4a5137d8-f1af-44f8-9288-ec13fcde8874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:51.008169 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8edcc-e808-4862-98c7-4c3ebacd1c98 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:51.016007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9099d8e-f614-4dad-91d7-501f7f1e1635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:51.016724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9099d8e-f614-4dad-91d7-501f7f1e1635 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:51.016868 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9099d8e-f614-4dad-91d7-501f7f1e1635 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:51.019998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9099d8e-f614-4dad-91d7-501f7f1e1635 admin admin] PATH_INFO: `/v3/projects/caf9864085c74207bd586b9493d91fe2/users/ae0dfb7613e1432dacbfc6011efaa0ae/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:51.044623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9099d8e-f614-4dad-91d7-501f7f1e1635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf9864085c74207bd586b9493d91fe2, user_id=ae0dfb7613e1432dacbfc6011efaa0ae, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:51.051139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9099d8e-f614-4dad-91d7-501f7f1e1635 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:51.085300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1034/2070] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:50 2022] PUT /identity/v3/projects/caf9864085c74207bd586b9493d91fe2/users/ae0dfb7613e1432dacbfc6011efaa0ae/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:51.090712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ed2fc-871d-4374-b83b-5f15d359505e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:51.091000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ed2fc-871d-4374-b83b-5f15d359505e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:51.091275 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ed2fc-871d-4374-b83b-5f15d359505e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:51.095341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5137d8-f1af-44f8-9288-ec13fcde8874 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:51.115369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-cd3ed2fc-871d-4374-b83b-5f15d359505e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:51.117105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-cd3ed2fc-871d-4374-b83b-5f15d359505e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:51.165023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe1341-9eb4-461a-b965-d13ecfc58bff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:51.165793 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe1341-9eb4-461a-b965-d13ecfc58bff service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:51.166603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe1341-9eb4-461a-b965-d13ecfc58bff service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:51.166867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe1341-9eb4-461a-b965-d13ecfc58bff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:51.220248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:51.230484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1035/2071] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:51.243377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ed2fc-871d-4374-b83b-5f15d359505e None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:51.249342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-cd3ed2fc-871d-4374-b83b-5f15d359505e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:51.254688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe1341-9eb4-461a-b965-d13ecfc58bff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:51.256227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe1341-9eb4-461a-b965-d13ecfc58bff service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:51.270192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1037/2072] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:51.311829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe1341-9eb4-461a-b965-d13ecfc58bff service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:51.317373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:51.323925 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe1341-9eb4-461a-b965-d13ecfc58bff service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:51.329002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:51.329656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:51.329937 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:51.330212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] PATH_INFO: `/v3/users/42bd91b43e65425ab1b37a13a65cb860` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:51.331808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42bd91b43e65425ab1b37a13a65cb860)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:51.333000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ed2fc-871d-4374-b83b-5f15d359505e None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:51.333789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:51.337508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb161f34-b108-4dc7-8c9e-df3e67824386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:51.338809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb161f34-b108-4dc7-8c9e-df3e67824386 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:51.339090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb161f34-b108-4dc7-8c9e-df3e67824386 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:51.339367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb161f34-b108-4dc7-8c9e-df3e67824386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:51.352938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb161f34-b108-4dc7-8c9e-df3e67824386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:51.355118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb161f34-b108-4dc7-8c9e-df3e67824386 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:51.385458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1036/2073] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:51.410491 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42bd91b43e65425ab1b37a13a65cb860'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:51.421157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42bd91b43e65425ab1b37a13a65cb860'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:51.422626 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42bd91b43e65425ab1b37a13a65cb860'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:51.444854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42bd91b43e65425ab1b37a13a65cb860'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:51.453329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42bd91b43e65425ab1b37a13a65cb860'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:51.464735 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1038/2074] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:51 2022] DELETE /identity/v3/users/42bd91b43e65425ab1b37a13a65cb860 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:51.467925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a8ad8-0458-4c04-8dd9-0fb5ed427ae9 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:51.468702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a8ad8-0458-4c04-8dd9-0fb5ed427ae9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:51.469405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a8ad8-0458-4c04-8dd9-0fb5ed427ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:51.499301 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a53a8ad8-0458-4c04-8dd9-0fb5ed427ae9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:51.500652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a53a8ad8-0458-4c04-8dd9-0fb5ed427ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:51.572563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1037/2075] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:51.583257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:51.585405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:51.667465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90095e4-f632-43ac-9634-19ec6f564c66 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:51.674371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d0c61-8d7b-44f4-a1c8-2aa6e210d0c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:51.675149 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d0c61-8d7b-44f4-a1c8-2aa6e210d0c3 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:51.675433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d0c61-8d7b-44f4-a1c8-2aa6e210d0c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:51.675724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d0c61-8d7b-44f4-a1c8-2aa6e210d0c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:51.689353 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d0c61-8d7b-44f4-a1c8-2aa6e210d0c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:51.690840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d0c61-8d7b-44f4-a1c8-2aa6e210d0c3 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:51.702292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1039/2076] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:52.026964 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a8ad8-0458-4c04-8dd9-0fb5ed427ae9 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:52.029113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a53a8ad8-0458-4c04-8dd9-0fb5ed427ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:52.083018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a8ad8-0458-4c04-8dd9-0fb5ed427ae9 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:52.092309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:52.094679 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:52.094965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:52.095233 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] PATH_INFO: `/v3/projects/c99341e13d5143588d11576bc20b5209` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:52.096671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99341e13d5143588d11576bc20b5209)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:52.098998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:52.204736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99341e13d5143588d11576bc20b5209', 'request_id': 'req-d020d213-808e-4883-9986-0a6e3815c784'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.214906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99341e13d5143588d11576bc20b5209', 'request_id': 'req-d020d213-808e-4883-9986-0a6e3815c784'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.231871 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99341e13d5143588d11576bc20b5209', 'request_id': 'req-d020d213-808e-4883-9986-0a6e3815c784'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.262635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] The token cache is being invalidate because project c99341e13d5143588d11576bc20b5209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:52.262928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.265623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.271601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1038/2077] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:52 2022] DELETE /identity/v3/projects/c99341e13d5143588d11576bc20b5209 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:52.276108 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d0c61-8d7b-44f4-a1c8-2aa6e210d0c3 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:52.297881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d0c61-8d7b-44f4-a1c8-2aa6e210d0c3 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:52.381733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:52.394117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345c69c-a4af-4038-8a96-0deb25635fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:52.395490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c69c-a4af-4038-8a96-0deb25635fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:52.397014 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c69c-a4af-4038-8a96-0deb25635fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:52.397969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c69c-a4af-4038-8a96-0deb25635fb5 admin admin] PATH_INFO: `/v3/users/fb81fdf05e8f44b4bd6f443ec6f0789b` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:52.399397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345c69c-a4af-4038-8a96-0deb25635fb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb81fdf05e8f44b4bd6f443ec6f0789b)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:52.402531 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345c69c-a4af-4038-8a96-0deb25635fb5 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:52.447608 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:52.450912 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:52.472224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020d213-808e-4883-9986-0a6e3815c784 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:52.479446 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59196e86-f95a-414a-854f-eeb638e33ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:52.480295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59196e86-f95a-414a-854f-eeb638e33ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:52.480650 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59196e86-f95a-414a-854f-eeb638e33ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:52.480994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59196e86-f95a-414a-854f-eeb638e33ea8 admin admin] PATH_INFO: `/v3/users/164a995758244cc0a4f89f1fb52fe281` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:52.482699 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59196e86-f95a-414a-854f-eeb638e33ea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=164a995758244cc0a4f89f1fb52fe281)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:52.485171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59196e86-f95a-414a-854f-eeb638e33ea8 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:52.505722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d345c69c-a4af-4038-8a96-0deb25635fb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb81fdf05e8f44b4bd6f443ec6f0789b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.523269 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d345c69c-a4af-4038-8a96-0deb25635fb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb81fdf05e8f44b4bd6f443ec6f0789b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.524265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d345c69c-a4af-4038-8a96-0deb25635fb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb81fdf05e8f44b4bd6f443ec6f0789b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.550802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d345c69c-a4af-4038-8a96-0deb25635fb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb81fdf05e8f44b4bd6f443ec6f0789b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.562292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d345c69c-a4af-4038-8a96-0deb25635fb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb81fdf05e8f44b4bd6f443ec6f0789b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.563885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-59196e86-f95a-414a-854f-eeb638e33ea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '164a995758244cc0a4f89f1fb52fe281'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.571289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1040/2078] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:52 2022] DELETE /identity/v3/users/fb81fdf05e8f44b4bd6f443ec6f0789b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:52.582408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-59196e86-f95a-414a-854f-eeb638e33ea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '164a995758244cc0a4f89f1fb52fe281'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.583482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-59196e86-f95a-414a-854f-eeb638e33ea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '164a995758244cc0a4f89f1fb52fe281'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.584827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-59196e86-f95a-414a-854f-eeb638e33ea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '164a995758244cc0a4f89f1fb52fe281'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.592224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:52.602881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-59196e86-f95a-414a-854f-eeb638e33ea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '164a995758244cc0a4f89f1fb52fe281'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:52.630722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1039/2079] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:52 2022] DELETE /identity/v3/users/164a995758244cc0a4f89f1fb52fe281 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:52.636617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb72c7c-f306-4685-a12d-b8dbb13dd911 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:52.637328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb72c7c-f306-4685-a12d-b8dbb13dd911 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:52.637856 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb72c7c-f306-4685-a12d-b8dbb13dd911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:52.664351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-9cb72c7c-f306-4685-a12d-b8dbb13dd911 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:52.665830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9cb72c7c-f306-4685-a12d-b8dbb13dd911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:52.722330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099587c5-ac15-4f19-914d-d6dd1403d053 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:52.722829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099587c5-ac15-4f19-914d-d6dd1403d053 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:52.723216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099587c5-ac15-4f19-914d-d6dd1403d053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:52.770447 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-099587c5-ac15-4f19-914d-d6dd1403d053 None None] MFA Rules not processed for user `ae0dfb7613e1432dacbfc6011efaa0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:52.773306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-099587c5-ac15-4f19-914d-d6dd1403d053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:52.782648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1041/2080] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:52.808720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb72c7c-f306-4685-a12d-b8dbb13dd911 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:52.811162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9cb72c7c-f306-4685-a12d-b8dbb13dd911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:52.903045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1040/2081] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:52.933732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099587c5-ac15-4f19-914d-d6dd1403d053 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:52.935542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-099587c5-ac15-4f19-914d-d6dd1403d053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:52.963205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb72c7c-f306-4685-a12d-b8dbb13dd911 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:52.972263 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aa679-3ed6-4c8d-81c2-f35f9aed71f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:52.973610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aa679-3ed6-4c8d-81c2-f35f9aed71f0 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:52.973896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aa679-3ed6-4c8d-81c2-f35f9aed71f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:52.974175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aa679-3ed6-4c8d-81c2-f35f9aed71f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:52.986199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aa679-3ed6-4c8d-81c2-f35f9aed71f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:52.987986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aa679-3ed6-4c8d-81c2-f35f9aed71f0 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:53.002618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1042/2082] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:53.029462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099587c5-ac15-4f19-914d-d6dd1403d053 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:53.034100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c8ef6-45d6-4556-a89e-70fafe503b51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:53.034703 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c8ef6-45d6-4556-a89e-70fafe503b51 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:53.035049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c8ef6-45d6-4556-a89e-70fafe503b51 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:53.035385 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c8ef6-45d6-4556-a89e-70fafe503b51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:53.048064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c8ef6-45d6-4556-a89e-70fafe503b51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:53.049785 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c8ef6-45d6-4556-a89e-70fafe503b51 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:53.061363 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1041/2083] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:53.370220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aa679-3ed6-4c8d-81c2-f35f9aed71f0 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:53.372156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-508aa679-3ed6-4c8d-81c2-f35f9aed71f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:53.393045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c8ef6-45d6-4556-a89e-70fafe503b51 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:53.412008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-eb9c8ef6-45d6-4556-a89e-70fafe503b51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:53.438641 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aa679-3ed6-4c8d-81c2-f35f9aed71f0 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:53.469182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c8ef6-45d6-4556-a89e-70fafe503b51 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:53.483635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:53.485516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:53.485670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:53.485785 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:53.486209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] PATH_INFO: `/v3/projects/ce05833464264433a3ec27827de46437` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:53.486339 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:53.486550 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:53.486830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37 admin admin] PATH_INFO: `/v3/projects/8d7b732ee777481c82f2b47e075d6390` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:53.487600 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce05833464264433a3ec27827de46437)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:53.488739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d7b732ee777481c82f2b47e075d6390)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:53.489775 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:53.492390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:53.514272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:53.593187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d7b732ee777481c82f2b47e075d6390', 'request_id': 'req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.600463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d7b732ee777481c82f2b47e075d6390', 'request_id': 'req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.605002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce05833464264433a3ec27827de46437', 'request_id': 'req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.613362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d7b732ee777481c82f2b47e075d6390', 'request_id': 'req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.617607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce05833464264433a3ec27827de46437', 'request_id': 'req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.633554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce05833464264433a3ec27827de46437', 'request_id': 'req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.634228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37 admin admin] The token cache is being invalidate because project 8d7b732ee777481c82f2b47e075d6390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:53.634782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.635982 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-34cb9127-cad9-4a36-bb6c-0fdaac4bda37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.637477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:53.638050 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1043/2084] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:53 2022] DELETE /identity/v3/projects/8d7b732ee777481c82f2b47e075d6390 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:53.652583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83109292-cad1-48d3-af97-2ad4995331aa None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:53.653578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83109292-cad1-48d3-af97-2ad4995331aa None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:53.653970 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83109292-cad1-48d3-af97-2ad4995331aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:53.657800 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] The token cache is being invalidate because project ce05833464264433a3ec27827de46437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:53.658099 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.658747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.660589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1042/2085] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:53 2022] DELETE /identity/v3/projects/ce05833464264433a3ec27827de46437 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:53.665355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:53.668452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:53.683032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-83109292-cad1-48d3-af97-2ad4995331aa None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:53.684545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-83109292-cad1-48d3-af97-2ad4995331aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:53.751302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc34118-ae5c-4ed5-817c-2e2a9b5307c4 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:53.759506 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1044/2086] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:53.767057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62644a8c-48d4-4a92-b213-61d3d183d3cc None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:53.767334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62644a8c-48d4-4a92-b213-61d3d183d3cc None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:53.767613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62644a8c-48d4-4a92-b213-61d3d183d3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:53.788709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-62644a8c-48d4-4a92-b213-61d3d183d3cc None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:53.791009 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-62644a8c-48d4-4a92-b213-61d3d183d3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:53.799833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1045/2087] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:53.803335 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62644a8c-48d4-4a92-b213-61d3d183d3cc None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:53.813362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62644a8c-48d4-4a92-b213-61d3d183d3cc None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:53.818604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1274b0-27d4-4f59-b4c7-c4f3887d588a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:53.819226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1274b0-27d4-4f59-b4c7-c4f3887d588a admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:53.819509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1274b0-27d4-4f59-b4c7-c4f3887d588a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:53.819810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1274b0-27d4-4f59-b4c7-c4f3887d588a admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:53.820597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1274b0-27d4-4f59-b4c7-c4f3887d588a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:53.820770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1274b0-27d4-4f59-b4c7-c4f3887d588a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:53.822405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1274b0-27d4-4f59-b4c7-c4f3887d588a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:53.829667 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:53.830384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:53.830664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:53.830933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] PATH_INFO: `/v3/users/5f45788f5b704eab89e8178f9ae31f68` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:53.832147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f45788f5b704eab89e8178f9ae31f68)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:53.834428 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:53.843143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1046/2088] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:52:53.847647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1274b0-27d4-4f59-b4c7-c4f3887d588a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:53.860810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1274b0-27d4-4f59-b4c7-c4f3887d588a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:53.866432 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d28793-95bc-4e64-864b-3db7e9ce9aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:53.867191 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d28793-95bc-4e64-864b-3db7e9ce9aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:53.867464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d28793-95bc-4e64-864b-3db7e9ce9aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:53.867806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d28793-95bc-4e64-864b-3db7e9ce9aa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:53.868442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d28793-95bc-4e64-864b-3db7e9ce9aa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:53.871323 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d28793-95bc-4e64-864b-3db7e9ce9aa8 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:53.906565 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f45788f5b704eab89e8178f9ae31f68'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.917791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f45788f5b704eab89e8178f9ae31f68'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.919085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f45788f5b704eab89e8178f9ae31f68'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.925583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f45788f5b704eab89e8178f9ae31f68'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.927851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1047/2089] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:53 2022] POST /identity/v3/projects => generated 409 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:53.929120 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d28793-95bc-4e64-864b-3db7e9ce9aa8 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:53.932003 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-78d28793-95bc-4e64-864b-3db7e9ce9aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:53.957159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f45788f5b704eab89e8178f9ae31f68'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:53.980339 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1043/2090] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:53 2022] DELETE /identity/v3/users/5f45788f5b704eab89e8178f9ae31f68 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:54.054938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:54.065211 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d28793-95bc-4e64-864b-3db7e9ce9aa8 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:54.159845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ffe8a-eb5b-4322-8a0e-5f79bfcacb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:54.160687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ffe8a-eb5b-4322-8a0e-5f79bfcacb98 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:54.161066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ffe8a-eb5b-4322-8a0e-5f79bfcacb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:54.161588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ffe8a-eb5b-4322-8a0e-5f79bfcacb98 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:54.162335 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ffe8a-eb5b-4322-8a0e-5f79bfcacb98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:54.164693 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ffe8a-eb5b-4322-8a0e-5f79bfcacb98 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:54.198593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1048/2091] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:53 2022] POST /identity/v3/users => generated 305 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:54.199280 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:54.217755 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190b856-9af9-4a9a-b9c9-31b421a9de35 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:54.232088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc3cb8-381e-4568-9069-0788e0137a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:54.232830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc3cb8-381e-4568-9069-0788e0137a66 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:54.233157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc3cb8-381e-4568-9069-0788e0137a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:54.233371 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc3cb8-381e-4568-9069-0788e0137a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:54.234171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc3cb8-381e-4568-9069-0788e0137a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:54.235717 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc3cb8-381e-4568-9069-0788e0137a66 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:54.257959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1044/2092] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:54.262413 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ffe8a-eb5b-4322-8a0e-5f79bfcacb98 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:54.275500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ffe8a-eb5b-4322-8a0e-5f79bfcacb98 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:54.281270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02c022-c90b-4838-be07-5201a2749797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:54.281980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02c022-c90b-4838-be07-5201a2749797 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:54.282444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02c022-c90b-4838-be07-5201a2749797 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:54.282928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02c022-c90b-4838-be07-5201a2749797 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:54.284105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02c022-c90b-4838-be07-5201a2749797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:54.287034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02c022-c90b-4838-be07-5201a2749797 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:54.302038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1049/2093] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:54.302795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc3cb8-381e-4568-9069-0788e0137a66 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:54.317255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc3cb8-381e-4568-9069-0788e0137a66 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:54.325853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790feced-fb03-427f-b2ad-998143417fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:54.326569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790feced-fb03-427f-b2ad-998143417fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:54.326902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790feced-fb03-427f-b2ad-998143417fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:54.327184 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790feced-fb03-427f-b2ad-998143417fe3 admin admin] PATH_INFO: `/v3/projects/e9d724c64c7c4dd7866ab7ba83bba2cb/users/567b0b553d6f4331a52bcb87c1e1736f/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:54.339472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:54.349084 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790feced-fb03-427f-b2ad-998143417fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9d724c64c7c4dd7866ab7ba83bba2cb, user_id=567b0b553d6f4331a52bcb87c1e1736f, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:54.353291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790feced-fb03-427f-b2ad-998143417fe3 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:54.378688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1045/2094] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:54 2022] PUT /identity/v3/projects/e9d724c64c7c4dd7866ab7ba83bba2cb/users/567b0b553d6f4331a52bcb87c1e1736f/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:54.391351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836719e-095a-4d78-9899-fe2039d50063 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:54.392185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836719e-095a-4d78-9899-fe2039d50063 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:54.392416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836719e-095a-4d78-9899-fe2039d50063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:54.416507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790feced-fb03-427f-b2ad-998143417fe3 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:54.418306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-790feced-fb03-427f-b2ad-998143417fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:54.485230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4836719e-095a-4d78-9899-fe2039d50063 None None] MFA Rules not processed for user `567b0b553d6f4331a52bcb87c1e1736f`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:54.486470 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4836719e-095a-4d78-9899-fe2039d50063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:54.577258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790feced-fb03-427f-b2ad-998143417fe3 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:54.596710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:54.597510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:54.598062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:54.598530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] PATH_INFO: `/v3/projects/1278ccbd9cea4496a10e13c2b014ea3c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:54.599952 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1278ccbd9cea4496a10e13c2b014ea3c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:54.603615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:54.631620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1050/2095] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:54.638389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbbf5f-75f2-41f7-a556-d5a9a2f6b96c None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:54.638868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbbf5f-75f2-41f7-a556-d5a9a2f6b96c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:54.639722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbbf5f-75f2-41f7-a556-d5a9a2f6b96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:54.659811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-cbcbbf5f-75f2-41f7-a556-d5a9a2f6b96c None None] MFA Rules not processed for user `ae0dfb7613e1432dacbfc6011efaa0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:54.661176 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-cbcbbf5f-75f2-41f7-a556-d5a9a2f6b96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:54.676115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1278ccbd9cea4496a10e13c2b014ea3c', 'request_id': 'req-554d1169-8958-4535-8d6e-40ab3a23b91f'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:54.684221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1278ccbd9cea4496a10e13c2b014ea3c', 'request_id': 'req-554d1169-8958-4535-8d6e-40ab3a23b91f'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:54.702768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1278ccbd9cea4496a10e13c2b014ea3c', 'request_id': 'req-554d1169-8958-4535-8d6e-40ab3a23b91f'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:54.721173 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] The token cache is being invalidate because project 1278ccbd9cea4496a10e13c2b014ea3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:54.721595 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:54.723408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:54.725277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1046/2096] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:54 2022] DELETE /identity/v3/projects/1278ccbd9cea4496a10e13c2b014ea3c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:54.726154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:54.727792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:54.760802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1051/2097] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:54.762022 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb5699-5000-4f8d-9d35-caf8cd663f93 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:54.762449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb5699-5000-4f8d-9d35-caf8cd663f93 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:54.762795 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb5699-5000-4f8d-9d35-caf8cd663f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:54.826104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-01bb5699-5000-4f8d-9d35-caf8cd663f93 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:54.827525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-01bb5699-5000-4f8d-9d35-caf8cd663f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:54.835400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d1169-8958-4535-8d6e-40ab3a23b91f admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:54.840926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1052/2098] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:54.849186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592a7be-9033-4abc-9491-b5b6d8e08628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:54.849186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592a7be-9033-4abc-9491-b5b6d8e08628 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:54.849186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592a7be-9033-4abc-9491-b5b6d8e08628 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:54.849186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592a7be-9033-4abc-9491-b5b6d8e08628 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:54.849701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592a7be-9033-4abc-9491-b5b6d8e08628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:54.849701 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592a7be-9033-4abc-9491-b5b6d8e08628 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:54.851163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00357a06-e7c8-45d8-8ce5-4fe38b148c6c None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:54.851522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00357a06-e7c8-45d8-8ce5-4fe38b148c6c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:54.851898 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00357a06-e7c8-45d8-8ce5-4fe38b148c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:54.882127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1047/2099] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:54 2022] POST /identity/v3/projects => generated 409 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:54.889267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592a7be-9033-4abc-9491-b5b6d8e08628 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:54.891584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-00357a06-e7c8-45d8-8ce5-4fe38b148c6c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:54.894343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:54.897410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-00357a06-e7c8-45d8-8ce5-4fe38b148c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:54.915774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:54.999320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592a7be-9033-4abc-9491-b5b6d8e08628 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:55.009299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a4cfa-113b-4e75-8f2b-25ee9cbcc6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:55.010685 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a4cfa-113b-4e75-8f2b-25ee9cbcc6fb admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.011284 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a4cfa-113b-4e75-8f2b-25ee9cbcc6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.011744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a4cfa-113b-4e75-8f2b-25ee9cbcc6fb admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.012559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a4cfa-113b-4e75-8f2b-25ee9cbcc6fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:55.015137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a4cfa-113b-4e75-8f2b-25ee9cbcc6fb admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:55.064988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1053/2100] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:55.066173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00357a06-e7c8-45d8-8ce5-4fe38b148c6c None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:55.074105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1048/2101] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:54 2022] POST /identity/v3/users => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:55.079042 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a4cfa-113b-4e75-8f2b-25ee9cbcc6fb admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:55.080731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00357a06-e7c8-45d8-8ce5-4fe38b148c6c None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:55.093846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617237c4-625f-47be-a916-652c2b4cc81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:55.094710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617237c4-625f-47be-a916-652c2b4cc81a admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.096183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617237c4-625f-47be-a916-652c2b4cc81a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.096183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617237c4-625f-47be-a916-652c2b4cc81a admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.098385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617237c4-625f-47be-a916-652c2b4cc81a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:55.098777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617237c4-625f-47be-a916-652c2b4cc81a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:55.101157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a4cfa-113b-4e75-8f2b-25ee9cbcc6fb admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:55.102046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617237c4-625f-47be-a916-652c2b4cc81a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:55.107404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78313a-eb4b-415b-9068-247644202d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:55.109119 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78313a-eb4b-415b-9068-247644202d2c admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.109119 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78313a-eb4b-415b-9068-247644202d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.109119 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78313a-eb4b-415b-9068-247644202d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.109668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78313a-eb4b-415b-9068-247644202d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:55.111896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78313a-eb4b-415b-9068-247644202d2c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:55.128397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1049/2102] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:55.129357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78313a-eb4b-415b-9068-247644202d2c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:55.130045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1054/2103] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:55.135595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617237c4-625f-47be-a916-652c2b4cc81a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:55.146969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78313a-eb4b-415b-9068-247644202d2c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:55.154661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617237c4-625f-47be-a916-652c2b4cc81a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:55.160116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd462fd0-ac5f-4e0f-a053-e99ede23a292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:55.160739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd462fd0-ac5f-4e0f-a053-e99ede23a292 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.161100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd462fd0-ac5f-4e0f-a053-e99ede23a292 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.161373 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd462fd0-ac5f-4e0f-a053-e99ede23a292 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.161795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b818150-2ef6-4149-a3bf-7c397e87bebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:55.162060 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd462fd0-ac5f-4e0f-a053-e99ede23a292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:55.163779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b818150-2ef6-4149-a3bf-7c397e87bebe admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.164238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd462fd0-ac5f-4e0f-a053-e99ede23a292 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:55.165663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b818150-2ef6-4149-a3bf-7c397e87bebe admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.166237 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b818150-2ef6-4149-a3bf-7c397e87bebe admin admin] PATH_INFO: `/v3/projects/91e3d9befb114cc0843454480b4eb547/users/073149702d76479f831b0ae746c18b5f/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.212888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b818150-2ef6-4149-a3bf-7c397e87bebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e3d9befb114cc0843454480b4eb547, user_id=073149702d76479f831b0ae746c18b5f, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:55.214907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1055/2104] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:55 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:55.217030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b818150-2ef6-4149-a3bf-7c397e87bebe admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:55.218781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd462fd0-ac5f-4e0f-a053-e99ede23a292 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:55.243057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1050/2105] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:55 2022] PUT /identity/v3/projects/91e3d9befb114cc0843454480b4eb547/users/073149702d76479f831b0ae746c18b5f/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:55.256098 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b818150-2ef6-4149-a3bf-7c397e87bebe admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:55.354547 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd462fd0-ac5f-4e0f-a053-e99ede23a292 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:55.367425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2144a-3448-40a1-9f0f-b9d15f9e573a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:55.373864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2144a-3448-40a1-9f0f-b9d15f9e573a admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.374619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2144a-3448-40a1-9f0f-b9d15f9e573a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.375102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2144a-3448-40a1-9f0f-b9d15f9e573a admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.376017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2144a-3448-40a1-9f0f-b9d15f9e573a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:55.380779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2144a-3448-40a1-9f0f-b9d15f9e573a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:55.382089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b818150-2ef6-4149-a3bf-7c397e87bebe admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:55.393798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f546ae8-ae44-4230-a5a1-6642e83f5ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:55.396896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f546ae8-ae44-4230-a5a1-6642e83f5ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.397348 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f546ae8-ae44-4230-a5a1-6642e83f5ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.397903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f546ae8-ae44-4230-a5a1-6642e83f5ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.398925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f546ae8-ae44-4230-a5a1-6642e83f5ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:55.401684 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f546ae8-ae44-4230-a5a1-6642e83f5ca4 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:55.421706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1051/2106] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:55.425901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f546ae8-ae44-4230-a5a1-6642e83f5ca4 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:55.429570 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1056/2107] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:55 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:55.435611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2144a-3448-40a1-9f0f-b9d15f9e573a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:55.439426 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f546ae8-ae44-4230-a5a1-6642e83f5ca4 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:55.444991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f8dcb-48d7-4471-aa1c-b35c1c274eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:55.447176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f8dcb-48d7-4471-aa1c-b35c1c274eb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.448225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f8dcb-48d7-4471-aa1c-b35c1c274eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.448761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f8dcb-48d7-4471-aa1c-b35c1c274eb0 admin admin] PATH_INFO: `/v3/projects/91e3d9befb114cc0843454480b4eb547/users/073149702d76479f831b0ae746c18b5f/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.450190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2144a-3448-40a1-9f0f-b9d15f9e573a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:55.455823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7f5c0-15c9-49f9-8893-ae736d54e8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:55.456927 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7f5c0-15c9-49f9-8893-ae736d54e8ce admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.457315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7f5c0-15c9-49f9-8893-ae736d54e8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.457661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7f5c0-15c9-49f9-8893-ae736d54e8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.458362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f8dcb-48d7-4471-aa1c-b35c1c274eb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e3d9befb114cc0843454480b4eb547, user_id=073149702d76479f831b0ae746c18b5f, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:55.458635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7f5c0-15c9-49f9-8893-ae736d54e8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:55.460434 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7f5c0-15c9-49f9-8893-ae736d54e8ce admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:55.475085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f8dcb-48d7-4471-aa1c-b35c1c274eb0 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:55.503742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1052/2108] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:55 2022] PUT /identity/v3/projects/91e3d9befb114cc0843454480b4eb547/users/073149702d76479f831b0ae746c18b5f/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:55.515464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1057/2109] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:55.516367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65879642-f8ee-4a6a-ad0b-e014fb79018c None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.516820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65879642-f8ee-4a6a-ad0b-e014fb79018c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.517161 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7f5c0-15c9-49f9-8893-ae736d54e8ce admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:55.517441 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65879642-f8ee-4a6a-ad0b-e014fb79018c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.544463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7f5c0-15c9-49f9-8893-ae736d54e8ce admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:55.578815 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-65879642-f8ee-4a6a-ad0b-e014fb79018c None None] MFA Rules not processed for user `073149702d76479f831b0ae746c18b5f`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:55.580627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-65879642-f8ee-4a6a-ad0b-e014fb79018c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:55.640596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:55.653761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:55.667225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab7ec4-eef7-441f-9c3f-fe1f826df1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:55.667225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab7ec4-eef7-441f-9c3f-fe1f826df1ca admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.667588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab7ec4-eef7-441f-9c3f-fe1f826df1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.667667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab7ec4-eef7-441f-9c3f-fe1f826df1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.672025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab7ec4-eef7-441f-9c3f-fe1f826df1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:55.673803 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab7ec4-eef7-441f-9c3f-fe1f826df1ca admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:55.696007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1058/2110] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:55.700627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab7ec4-eef7-441f-9c3f-fe1f826df1ca admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:55.718071 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab7ec4-eef7-441f-9c3f-fe1f826df1ca admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:55.727730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1053/2111] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:55 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:55.742412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65879642-f8ee-4a6a-ad0b-e014fb79018c None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:55.744807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-65879642-f8ee-4a6a-ad0b-e014fb79018c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:55.745627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec8b31-a942-4d7f-a2c1-b78346438b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:55.747845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec8b31-a942-4d7f-a2c1-b78346438b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.748259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec8b31-a942-4d7f-a2c1-b78346438b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.748656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec8b31-a942-4d7f-a2c1-b78346438b3f admin admin] PATH_INFO: `/v3/projects/eac4b53b17ec466a84ab04fa38b49627/users/1ee7f5d2dac94797ad7c6681b6cffcd5/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.788557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec8b31-a942-4d7f-a2c1-b78346438b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eac4b53b17ec466a84ab04fa38b49627, user_id=1ee7f5d2dac94797ad7c6681b6cffcd5, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:55.793894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec8b31-a942-4d7f-a2c1-b78346438b3f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:55.821566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1059/2112] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:55 2022] PUT /identity/v3/projects/eac4b53b17ec466a84ab04fa38b49627/users/1ee7f5d2dac94797ad7c6681b6cffcd5/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:55.832780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f18d97-cc26-4d9e-a909-dc4557aa8fbb None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:55.833061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f18d97-cc26-4d9e-a909-dc4557aa8fbb None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:55.833375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f18d97-cc26-4d9e-a909-dc4557aa8fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:55.891730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65879642-f8ee-4a6a-ad0b-e014fb79018c None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:55.948536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-b9f18d97-cc26-4d9e-a909-dc4557aa8fbb None None] MFA Rules not processed for user `1ee7f5d2dac94797ad7c6681b6cffcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:55.949810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b9f18d97-cc26-4d9e-a909-dc4557aa8fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:56.041986 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce1042-2b96-4910-8dd3-649120061fa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:56.042660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce1042-2b96-4910-8dd3-649120061fa2 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:56.046398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce1042-2b96-4910-8dd3-649120061fa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:56.047887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce1042-2b96-4910-8dd3-649120061fa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:56.213460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce1042-2b96-4910-8dd3-649120061fa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:56.217755 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1060/2113] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:56.223774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f18d97-cc26-4d9e-a909-dc4557aa8fbb None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:56.230379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce1042-2b96-4910-8dd3-649120061fa2 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:56.262307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1054/2114] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:55 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:56.407734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f18d97-cc26-4d9e-a909-dc4557aa8fbb None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:56.407734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b3b04-7d85-42a6-952b-202eb8f32ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:56.407734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b3b04-7d85-42a6-952b-202eb8f32ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:56.407734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b3b04-7d85-42a6-952b-202eb8f32ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:56.408414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b3b04-7d85-42a6-952b-202eb8f32ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:56.408663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b3b04-7d85-42a6-952b-202eb8f32ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:56.412990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b3b04-7d85-42a6-952b-202eb8f32ed0 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:56.457407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1061/2115] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:56 2022] POST /identity/v3/projects => generated 375 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:56.459198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce1042-2b96-4910-8dd3-649120061fa2 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:56.480165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce1042-2b96-4910-8dd3-649120061fa2 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:56.570645 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:56.652416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea6f33-acd1-479c-ae87-8298094d8047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:56.653989 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea6f33-acd1-479c-ae87-8298094d8047 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:56.654761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea6f33-acd1-479c-ae87-8298094d8047 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:56.655671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea6f33-acd1-479c-ae87-8298094d8047 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:56.662410 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea6f33-acd1-479c-ae87-8298094d8047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:56.662410 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea6f33-acd1-479c-ae87-8298094d8047 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:56.671035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b3b04-7d85-42a6-952b-202eb8f32ed0 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:56.696268 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1055/2116] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:56 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:56.699396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea6f33-acd1-479c-ae87-8298094d8047 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:56.714340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea6f33-acd1-479c-ae87-8298094d8047 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:56.736431 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:56.739905 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690215d-02d9-4efc-96ad-0f519d6a9e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:56.740682 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690215d-02d9-4efc-96ad-0f519d6a9e6e admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:56.740981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690215d-02d9-4efc-96ad-0f519d6a9e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:56.741325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690215d-02d9-4efc-96ad-0f519d6a9e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:56.741978 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690215d-02d9-4efc-96ad-0f519d6a9e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:56.745397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690215d-02d9-4efc-96ad-0f519d6a9e6e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:56.753372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b3b04-7d85-42a6-952b-202eb8f32ed0 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:56.758403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066619ca-1ad0-4b67-812e-9f842a5ba0a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:56.759471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1056/2117] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:56.763707 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066619ca-1ad0-4b67-812e-9f842a5ba0a9 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:56.764192 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690215d-02d9-4efc-96ad-0f519d6a9e6e admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:56.767204 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066619ca-1ad0-4b67-812e-9f842a5ba0a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:56.767495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066619ca-1ad0-4b67-812e-9f842a5ba0a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:56.770903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-066619ca-1ad0-4b67-812e-9f842a5ba0a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:56.781303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690215d-02d9-4efc-96ad-0f519d6a9e6e admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:56.798132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18a6da-3c6c-4356-9c15-45dd46bf6d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:56.799008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18a6da-3c6c-4356-9c15-45dd46bf6d05 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:56.803585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18a6da-3c6c-4356-9c15-45dd46bf6d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:56.803951 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18a6da-3c6c-4356-9c15-45dd46bf6d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:56.804652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18a6da-3c6c-4356-9c15-45dd46bf6d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:56.807226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18a6da-3c6c-4356-9c15-45dd46bf6d05 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:56.840055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1057/2118] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:56.844300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18a6da-3c6c-4356-9c15-45dd46bf6d05 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:56.859127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18a6da-3c6c-4356-9c15-45dd46bf6d05 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:56.865049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f8e07-bf41-4553-bf58-fae8e30873d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:56.865828 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f8e07-bf41-4553-bf58-fae8e30873d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:56.866261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f8e07-bf41-4553-bf58-fae8e30873d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:56.866572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f8e07-bf41-4553-bf58-fae8e30873d0 admin admin] PATH_INFO: `/v3/projects/7fda93032fd64612b3bfbe450524664d/users/471a0670e24f4320b4923160bf9621bb/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:56.885945 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f8e07-bf41-4553-bf58-fae8e30873d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fda93032fd64612b3bfbe450524664d, user_id=471a0670e24f4320b4923160bf9621bb, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:56.888896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f8e07-bf41-4553-bf58-fae8e30873d0 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:56.901987 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066619ca-1ad0-4b67-812e-9f842a5ba0a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:56.903440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066619ca-1ad0-4b67-812e-9f842a5ba0a9 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:56.916932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1058/2119] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:56 2022] PUT /identity/v3/projects/7fda93032fd64612b3bfbe450524664d/users/471a0670e24f4320b4923160bf9621bb/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:56.918269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd35052-f3b2-41d1-9f6b-bd77f13b4fad None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:56.918555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd35052-f3b2-41d1-9f6b-bd77f13b4fad None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:56.918831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd35052-f3b2-41d1-9f6b-bd77f13b4fad None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:56.921272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1059/2120] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 14:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:52:56.928500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03659a6c-b176-4fee-8c81-697a902015be None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:56.928815 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03659a6c-b176-4fee-8c81-697a902015be None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:56.929049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03659a6c-b176-4fee-8c81-697a902015be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:56.931756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1062/2121] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:56 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:56.956314 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fe82f-b9fd-4dd6-9b06-cd96779cfa82 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:56.956706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fe82f-b9fd-4dd6-9b06-cd96779cfa82 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:56.956873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fe82f-b9fd-4dd6-9b06-cd96779cfa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:56.993600 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-0d0fe82f-b9fd-4dd6-9b06-cd96779cfa82 None None] MFA Rules not processed for user `db581b10d78a4f9eae33fadce0d92b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:56.994783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0d0fe82f-b9fd-4dd6-9b06-cd96779cfa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:56.998278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-03659a6c-b176-4fee-8c81-697a902015be None None] MFA Rules not processed for user `471a0670e24f4320b4923160bf9621bb`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:56.999566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-03659a6c-b176-4fee-8c81-697a902015be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:57.115748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1063/2122] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 14:52:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:57.122702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fe82f-b9fd-4dd6-9b06-cd96779cfa82 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:57.124339 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0d0fe82f-b9fd-4dd6-9b06-cd96779cfa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:57.141676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1060/2123] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:57.141676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03659a6c-b176-4fee-8c81-697a902015be None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:57.146870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-03659a6c-b176-4fee-8c81-697a902015be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:57.162943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03659a6c-b176-4fee-8c81-697a902015be None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:57.170384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595149a8-d59e-48dc-881e-e5b0a4d3c0a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:57.171114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595149a8-d59e-48dc-881e-e5b0a4d3c0a1 service placement] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:57.171631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595149a8-d59e-48dc-881e-e5b0a4d3c0a1 service placement] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:57.172045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595149a8-d59e-48dc-881e-e5b0a4d3c0a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:57.189736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595149a8-d59e-48dc-881e-e5b0a4d3c0a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:57.191404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595149a8-d59e-48dc-881e-e5b0a4d3c0a1 service placement] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:57.210339 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1061/2124] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:52:57.218477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595149a8-d59e-48dc-881e-e5b0a4d3c0a1 service placement] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:57.259124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fe82f-b9fd-4dd6-9b06-cd96779cfa82 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:57.268770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:57.270845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:57.271401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:57.273765 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] PATH_INFO: `/v3/users/1469a960bd9241008cfb112a28bcd5d3` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:57.279371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1469a960bd9241008cfb112a28bcd5d3)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:57.282704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:57.319883 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:57.325397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:57.341902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595149a8-d59e-48dc-881e-e5b0a4d3c0a1 service placement] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:57.345910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1469a960bd9241008cfb112a28bcd5d3'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:57.358443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1469a960bd9241008cfb112a28bcd5d3'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:57.359232 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1469a960bd9241008cfb112a28bcd5d3'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:57.360368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bd21a-d264-4cf8-9f0f-bd813f686af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:57.362137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bd21a-d264-4cf8-9f0f-bd813f686af1 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:57.362712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bd21a-d264-4cf8-9f0f-bd813f686af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:57.363241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bd21a-d264-4cf8-9f0f-bd813f686af1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:57.364269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bd21a-d264-4cf8-9f0f-bd813f686af1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:57.367634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bd21a-d264-4cf8-9f0f-bd813f686af1 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:57.389469 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1469a960bd9241008cfb112a28bcd5d3'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:57.405092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1469a960bd9241008cfb112a28bcd5d3'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:57.422254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1062/2125] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:57 2022] POST /identity/v3/projects => generated 375 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:57.427477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bd21a-d264-4cf8-9f0f-bd813f686af1 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:57.430061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1f1bd21a-d264-4cf8-9f0f-bd813f686af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:57.433159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1064/2126] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:52:57 2022] DELETE /identity/v3/users/1469a960bd9241008cfb112a28bcd5d3 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:57.567788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bd21a-d264-4cf8-9f0f-bd813f686af1 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:57.633414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9558b95-4430-46b7-b258-062558da9e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:57.634167 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9558b95-4430-46b7-b258-062558da9e59 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:57.634502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9558b95-4430-46b7-b258-062558da9e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:57.635391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9558b95-4430-46b7-b258-062558da9e59 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:57.636132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9558b95-4430-46b7-b258-062558da9e59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:57.638137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9558b95-4430-46b7-b258-062558da9e59 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:57.662802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1063/2127] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:57 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:57.664371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:57.677942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9558b95-4430-46b7-b258-062558da9e59 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:57.679810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e9558b95-4430-46b7-b258-062558da9e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:57.680831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25274fc6-424f-4d86-ba16-ea693679cb9c admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:57.688202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56f413-9785-4129-8734-513ecc5f9130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:57.689132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56f413-9785-4129-8734-513ecc5f9130 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:57.689255 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56f413-9785-4129-8734-513ecc5f9130 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:57.689590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56f413-9785-4129-8734-513ecc5f9130 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:57.690413 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56f413-9785-4129-8734-513ecc5f9130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:57.693241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9558b95-4430-46b7-b258-062558da9e59 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:57.693380 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56f413-9785-4129-8734-513ecc5f9130 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:57.700716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:57.701372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:57.701641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:57.701911 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1 admin admin] PATH_INFO: `/v3/projects/2a38dab41aca4c2d9258599840ce93d5` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:57.703088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a38dab41aca4c2d9258599840ce93d5)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:57.705187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:57.706440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1065/2128] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:52:57.710932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56f413-9785-4129-8734-513ecc5f9130 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:57.721862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56f413-9785-4129-8734-513ecc5f9130 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:57.726485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2efed-5b4c-4755-a909-ab8fb29525c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:57.727389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2efed-5b4c-4755-a909-ab8fb29525c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:57.727548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2efed-5b4c-4755-a909-ab8fb29525c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:57.727870 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2efed-5b4c-4755-a909-ab8fb29525c1 admin admin] PATH_INFO: `/v3/projects/6e63f8b24dc34b4aa45c45a03acb6968/users/64f6aea849c3480fb70afc3a9e5f9529/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:57.744933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2efed-5b4c-4755-a909-ab8fb29525c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e63f8b24dc34b4aa45c45a03acb6968, user_id=64f6aea849c3480fb70afc3a9e5f9529, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:57.747638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2efed-5b4c-4755-a909-ab8fb29525c1 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:57.763170 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1066/2129] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:57 2022] PUT /identity/v3/projects/6e63f8b24dc34b4aa45c45a03acb6968/users/64f6aea849c3480fb70afc3a9e5f9529/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:52:57.763600 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a38dab41aca4c2d9258599840ce93d5', 'request_id': 'req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:57.768083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2efed-5b4c-4755-a909-ab8fb29525c1 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:57.769696 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a38dab41aca4c2d9258599840ce93d5', 'request_id': 'req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:57.778921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a38dab41aca4c2d9258599840ce93d5', 'request_id': 'req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:57.781148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:57.795405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1 admin admin] The token cache is being invalidate because project 2a38dab41aca4c2d9258599840ce93d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:52:57.795816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:57.796817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ae7aad6a-85e1-406c-9b61-acb6bc60d8b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:52:57.799196 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1064/2130] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:52:57 2022] DELETE /identity/v3/projects/2a38dab41aca4c2d9258599840ce93d5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:57.803962 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853085c-69f3-44ca-b02b-93d0cfe58e63 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:57.804314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853085c-69f3-44ca-b02b-93d0cfe58e63 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:57.804660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853085c-69f3-44ca-b02b-93d0cfe58e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:57.828368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-e853085c-69f3-44ca-b02b-93d0cfe58e63 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:57.829604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e853085c-69f3-44ca-b02b-93d0cfe58e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:57.852924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2efed-5b4c-4755-a909-ab8fb29525c1 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:57.861956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a355cad-57de-4ecc-8aa1-608ff0620eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:57.862661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a355cad-57de-4ecc-8aa1-608ff0620eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:57.863001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a355cad-57de-4ecc-8aa1-608ff0620eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:57.863337 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a355cad-57de-4ecc-8aa1-608ff0620eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:57.864104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a355cad-57de-4ecc-8aa1-608ff0620eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:57.866211 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a355cad-57de-4ecc-8aa1-608ff0620eb6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:57.883481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1067/2131] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:57.884882 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a355cad-57de-4ecc-8aa1-608ff0620eb6 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:57.887126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3a355cad-57de-4ecc-8aa1-608ff0620eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:57.903885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a355cad-57de-4ecc-8aa1-608ff0620eb6 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:57.907456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1065/2132] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:57.915565 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f158bf8-71e9-4c32-902d-13337204cf72 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:57.916496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f158bf8-71e9-4c32-902d-13337204cf72 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:57.916682 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f158bf8-71e9-4c32-902d-13337204cf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:57.924582 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766ddf1-189a-4f23-8870-57e4d33e3f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:57.926464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766ddf1-189a-4f23-8870-57e4d33e3f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:57.926761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766ddf1-189a-4f23-8870-57e4d33e3f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:57.927030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766ddf1-189a-4f23-8870-57e4d33e3f69 admin admin] PATH_INFO: `/v3/projects/6e63f8b24dc34b4aa45c45a03acb6968/users/64f6aea849c3480fb70afc3a9e5f9529/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:57.932575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766ddf1-189a-4f23-8870-57e4d33e3f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e63f8b24dc34b4aa45c45a03acb6968, user_id=64f6aea849c3480fb70afc3a9e5f9529, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:57.935419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766ddf1-189a-4f23-8870-57e4d33e3f69 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:57.957243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1068/2133] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:57 2022] PUT /identity/v3/projects/6e63f8b24dc34b4aa45c45a03acb6968/users/64f6aea849c3480fb70afc3a9e5f9529/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:57.957471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-2f158bf8-71e9-4c32-902d-13337204cf72 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:57.957471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2f158bf8-71e9-4c32-902d-13337204cf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:57.960632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37b1f2-055f-4676-ab99-d6d5578678bd None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:57.961019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37b1f2-055f-4676-ab99-d6d5578678bd None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:57.961334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37b1f2-055f-4676-ab99-d6d5578678bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:57.997447 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-cd37b1f2-055f-4676-ab99-d6d5578678bd None None] MFA Rules not processed for user `64f6aea849c3480fb70afc3a9e5f9529`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:58.000128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-cd37b1f2-055f-4676-ab99-d6d5578678bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:58.064148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1066/2134] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:58.065441 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f158bf8-71e9-4c32-902d-13337204cf72 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:58.074639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:58.076218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f158bf8-71e9-4c32-902d-13337204cf72 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:58.083974 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f573295-253a-4f45-b046-f518202d8fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:58.084897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f573295-253a-4f45-b046-f518202d8fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:58.085400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f573295-253a-4f45-b046-f518202d8fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:58.086086 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f573295-253a-4f45-b046-f518202d8fe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:58.086986 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f573295-253a-4f45-b046-f518202d8fe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:52:58.087392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f573295-253a-4f45-b046-f518202d8fe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:58.090497 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f573295-253a-4f45-b046-f518202d8fe8 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:58.121822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1069/2135] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:52:58.125668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1067/2136] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:52:58.131941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37b1f2-055f-4676-ab99-d6d5578678bd None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:58.134297 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f573295-253a-4f45-b046-f518202d8fe8 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:58.142734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1f573295-253a-4f45-b046-f518202d8fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:58.144547 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37b1f2-055f-4676-ab99-d6d5578678bd None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:58.154430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9b4f3-ce69-4972-a8a8-6de13c6a9721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:58.154910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9b4f3-ce69-4972-a8a8-6de13c6a9721 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:58.156069 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9b4f3-ce69-4972-a8a8-6de13c6a9721 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:58.156475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9b4f3-ce69-4972-a8a8-6de13c6a9721 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:58.157951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9b4f3-ce69-4972-a8a8-6de13c6a9721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:58.162794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9b4f3-ce69-4972-a8a8-6de13c6a9721 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:58.202068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1070/2137] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:52:58 2022] POST /identity/v3/projects => generated 393 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:52:58.206649 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9b4f3-ce69-4972-a8a8-6de13c6a9721 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:58.223221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f573295-253a-4f45-b046-f518202d8fe8 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:58.272256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:52:58.286760 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac075584-7628-449a-8e63-911eedede8da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:58.287308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac075584-7628-449a-8e63-911eedede8da service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:58.287597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac075584-7628-449a-8e63-911eedede8da service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:58.287965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac075584-7628-449a-8e63-911eedede8da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:58.289825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ac075584-7628-449a-8e63-911eedede8da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:58.302570 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9b4f3-ce69-4972-a8a8-6de13c6a9721 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:58.307734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013094ae-fd59-4949-955d-9b5091660ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:58.308499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013094ae-fd59-4949-955d-9b5091660ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:58.308917 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013094ae-fd59-4949-955d-9b5091660ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:58.309181 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013094ae-fd59-4949-955d-9b5091660ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:58.310665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013094ae-fd59-4949-955d-9b5091660ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:58.312125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013094ae-fd59-4949-955d-9b5091660ca6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:58.340870 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1071/2138] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:52:58 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:52:58.342820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013094ae-fd59-4949-955d-9b5091660ca6 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:58.354922 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013094ae-fd59-4949-955d-9b5091660ca6 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:58.360459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e157f-2fc4-4040-b038-b885ba01cf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:58.361148 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e157f-2fc4-4040-b038-b885ba01cf4d admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:58.361497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e157f-2fc4-4040-b038-b885ba01cf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:58.361880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e157f-2fc4-4040-b038-b885ba01cf4d admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:58.362574 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e157f-2fc4-4040-b038-b885ba01cf4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:58.364304 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e157f-2fc4-4040-b038-b885ba01cf4d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:58.370168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac075584-7628-449a-8e63-911eedede8da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:58.371740 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac075584-7628-449a-8e63-911eedede8da service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:58.377410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1072/2139] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:58.380157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e157f-2fc4-4040-b038-b885ba01cf4d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:58.386779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1068/2140] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:52:58.391306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e157f-2fc4-4040-b038-b885ba01cf4d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:58.397142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de8b78-0a19-435c-8eff-30fc6ef9b49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:58.397838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de8b78-0a19-435c-8eff-30fc6ef9b49d admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:58.398190 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de8b78-0a19-435c-8eff-30fc6ef9b49d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:58.398536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de8b78-0a19-435c-8eff-30fc6ef9b49d admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:58.399262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de8b78-0a19-435c-8eff-30fc6ef9b49d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:58.401708 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de8b78-0a19-435c-8eff-30fc6ef9b49d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:58.417148 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1073/2141] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:52:58.419721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac075584-7628-449a-8e63-911eedede8da service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:58.439492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac075584-7628-449a-8e63-911eedede8da service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:58.449195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8db413-3f88-4d6b-a02b-8fc38083138a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:58.449890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8db413-3f88-4d6b-a02b-8fc38083138a admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:58.450194 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8db413-3f88-4d6b-a02b-8fc38083138a admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:58.450687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8db413-3f88-4d6b-a02b-8fc38083138a admin admin] PATH_INFO: `/v3/projects/acc902aa109c4d7291c0bab62f80048e/users/26eaa7f102bd41028983458ae48ff992/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:58.483934 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8db413-3f88-4d6b-a02b-8fc38083138a admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc902aa109c4d7291c0bab62f80048e, user_id=26eaa7f102bd41028983458ae48ff992, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:58.487744 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8db413-3f88-4d6b-a02b-8fc38083138a admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:58.506898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1069/2142] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:52:58 2022] PUT /identity/v3/projects/acc902aa109c4d7291c0bab62f80048e/users/26eaa7f102bd41028983458ae48ff992/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:52:58.512988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a5f3f-588f-457c-9a0f-a4e8827a9d36 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:58.513302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a5f3f-588f-457c-9a0f-a4e8827a9d36 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:58.513577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a5f3f-588f-457c-9a0f-a4e8827a9d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:58.542295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-685a5f3f-588f-457c-9a0f-a4e8827a9d36 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:52:58.544313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-685a5f3f-588f-457c-9a0f-a4e8827a9d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:58.678249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1074/2143] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:52:58.702304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8db413-3f88-4d6b-a02b-8fc38083138a admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:52:58.705397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ee8db413-3f88-4d6b-a02b-8fc38083138a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:52:58.810397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8db413-3f88-4d6b-a02b-8fc38083138a admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:52:58.817633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa9eef-f2ba-4b8d-9987-4e2ca72915f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:52:58.818855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa9eef-f2ba-4b8d-9987-4e2ca72915f3 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:52:58.819385 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa9eef-f2ba-4b8d-9987-4e2ca72915f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:52:58.820145 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa9eef-f2ba-4b8d-9987-4e2ca72915f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:52:58.870976 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa9eef-f2ba-4b8d-9987-4e2ca72915f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:52:58.872641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa9eef-f2ba-4b8d-9987-4e2ca72915f3 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:52:58.903230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1070/2144] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:03.540955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a5f3f-588f-457c-9a0f-a4e8827a9d36 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:03.542971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-685a5f3f-588f-457c-9a0f-a4e8827a9d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:03.600110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:03.645949 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a5f3f-588f-457c-9a0f-a4e8827a9d36 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:03.652810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a867a16-74e5-40f9-aca1-760dacc1efa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:03.653446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a867a16-74e5-40f9-aca1-760dacc1efa4 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:03.653732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a867a16-74e5-40f9-aca1-760dacc1efa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:03.654002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a867a16-74e5-40f9-aca1-760dacc1efa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:03.655946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4a867a16-74e5-40f9-aca1-760dacc1efa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:03.791036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a867a16-74e5-40f9-aca1-760dacc1efa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:03.797110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a867a16-74e5-40f9-aca1-760dacc1efa4 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:03.821408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1075/2145] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:04.354714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6a660-d028-48f3-b055-d0f87e5e30bd None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:04.355901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6a660-d028-48f3-b055-d0f87e5e30bd None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:04.356487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6a660-d028-48f3-b055-d0f87e5e30bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:04.396544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-3bf6a660-d028-48f3-b055-d0f87e5e30bd None None] MFA Rules not processed for user `26eaa7f102bd41028983458ae48ff992`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:04.397883 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3bf6a660-d028-48f3-b055-d0f87e5e30bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:04.512043 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1071/2146] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:04.526100 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbc9db-0498-4f78-8818-dfeb1b89be04 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:04.526496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbc9db-0498-4f78-8818-dfeb1b89be04 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:04.526740 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbc9db-0498-4f78-8818-dfeb1b89be04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:04.549747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-30bbc9db-0498-4f78-8818-dfeb1b89be04 None None] MFA Rules not processed for user `26eaa7f102bd41028983458ae48ff992`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:04.551216 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-30bbc9db-0498-4f78-8818-dfeb1b89be04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:04.564932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1076/2147] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:04.585092 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6a660-d028-48f3-b055-d0f87e5e30bd None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:04.586338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3bf6a660-d028-48f3-b055-d0f87e5e30bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:04.597175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:04.691880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6a660-d028-48f3-b055-d0f87e5e30bd None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:04.701137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d33bf-fb9f-4d5d-8da7-3964e5b92b01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:04.701772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d33bf-fb9f-4d5d-8da7-3964e5b92b01 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:04.702055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d33bf-fb9f-4d5d-8da7-3964e5b92b01 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:04.702329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d33bf-fb9f-4d5d-8da7-3964e5b92b01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:04.714968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d33bf-fb9f-4d5d-8da7-3964e5b92b01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:04.716836 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d33bf-fb9f-4d5d-8da7-3964e5b92b01 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:04.718610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbc9db-0498-4f78-8818-dfeb1b89be04 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:04.727003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1072/2148] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:04 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:04.728988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbc9db-0498-4f78-8818-dfeb1b89be04 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:04.736273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03837e8c-a23e-4f56-9bd9-a5c7be88f852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:04.737196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837e8c-a23e-4f56-9bd9-a5c7be88f852 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:04.737196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837e8c-a23e-4f56-9bd9-a5c7be88f852 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:04.737438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837e8c-a23e-4f56-9bd9-a5c7be88f852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:04.820035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03837e8c-a23e-4f56-9bd9-a5c7be88f852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:04.821526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03837e8c-a23e-4f56-9bd9-a5c7be88f852 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:04.834965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1077/2149] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:53:05.563939 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03ffc7-7da5-413e-9d9e-9476683f6bcb None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:05.564705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03ffc7-7da5-413e-9d9e-9476683f6bcb None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:05.564705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03ffc7-7da5-413e-9d9e-9476683f6bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:05.585926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-7f03ffc7-7da5-413e-9d9e-9476683f6bcb None None] MFA Rules not processed for user `26eaa7f102bd41028983458ae48ff992`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:05.587174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7f03ffc7-7da5-413e-9d9e-9476683f6bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:05.600153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1073/2150] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:06.506711 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03837e8c-a23e-4f56-9bd9-a5c7be88f852 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:06.508496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-03837e8c-a23e-4f56-9bd9-a5c7be88f852 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:06.535260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03837e8c-a23e-4f56-9bd9-a5c7be88f852 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:06.549429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:06.550099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:06.550379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:06.550651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] PATH_INFO: `/v3/users/567b0b553d6f4331a52bcb87c1e1736f` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:06.552398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] RBAC: Authorizing `identity:delete_user(user_id=567b0b553d6f4331a52bcb87c1e1736f)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:06.567653 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:06.735279 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '567b0b553d6f4331a52bcb87c1e1736f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:06.750511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '567b0b553d6f4331a52bcb87c1e1736f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:06.752162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '567b0b553d6f4331a52bcb87c1e1736f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:06.765968 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:06.794464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '567b0b553d6f4331a52bcb87c1e1736f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:06.816680 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '567b0b553d6f4331a52bcb87c1e1736f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:06.832130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1078/2151] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:53:06 2022] DELETE /identity/v3/users/567b0b553d6f4331a52bcb87c1e1736f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:06.835094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22db44-d10d-4c4d-bab1-2d8dab90c0eb None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:06.835578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22db44-d10d-4c4d-bab1-2d8dab90c0eb None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:06.835713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22db44-d10d-4c4d-bab1-2d8dab90c0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:06.874812 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-ac22db44-d10d-4c4d-bab1-2d8dab90c0eb None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:06.876189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ac22db44-d10d-4c4d-bab1-2d8dab90c0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:06.983232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1074/2152] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:06.993851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:06.998525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:07.117294 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf6cf8-7884-4784-b257-00d76948f19c admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:07.124882 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf663505-66d4-4a84-8b4c-97a34cf11e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:07.126120 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf663505-66d4-4a84-8b4c-97a34cf11e72 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:07.126686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf663505-66d4-4a84-8b4c-97a34cf11e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:07.127130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf663505-66d4-4a84-8b4c-97a34cf11e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:07.143620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf663505-66d4-4a84-8b4c-97a34cf11e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:07.145488 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf663505-66d4-4a84-8b4c-97a34cf11e72 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:07.178257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1079/2153] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:07.836899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22db44-d10d-4c4d-bab1-2d8dab90c0eb None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:07.840942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ac22db44-d10d-4c4d-bab1-2d8dab90c0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:07.867589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22db44-d10d-4c4d-bab1-2d8dab90c0eb None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:07.876501 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:07.877171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:07.878083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:07.878367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] PATH_INFO: `/v3/projects/e9d724c64c7c4dd7866ab7ba83bba2cb` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:07.880522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9d724c64c7c4dd7866ab7ba83bba2cb)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:07.882477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:08.020603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9d724c64c7c4dd7866ab7ba83bba2cb', 'request_id': 'req-d1ebc08a-9bdf-4994-909f-ed33c90d5007'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:08.045640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9d724c64c7c4dd7866ab7ba83bba2cb', 'request_id': 'req-d1ebc08a-9bdf-4994-909f-ed33c90d5007'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:08.081852 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9d724c64c7c4dd7866ab7ba83bba2cb', 'request_id': 'req-d1ebc08a-9bdf-4994-909f-ed33c90d5007'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:08.105507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] The token cache is being invalidate because project e9d724c64c7c4dd7866ab7ba83bba2cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:53:08.106888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:08.109014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:08.112287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1075/2154] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:53:07 2022] DELETE /identity/v3/projects/e9d724c64c7c4dd7866ab7ba83bba2cb => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:08.116928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf663505-66d4-4a84-8b4c-97a34cf11e72 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:08.141582 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf663505-66d4-4a84-8b4c-97a34cf11e72 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:08.307562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0045f3-9043-4816-bb39-301148567ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:08.308254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0045f3-9043-4816-bb39-301148567ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:08.311574 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0045f3-9043-4816-bb39-301148567ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:08.311930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0045f3-9043-4816-bb39-301148567ff6 admin admin] PATH_INFO: `/v3/users/073149702d76479f831b0ae746c18b5f` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:08.315991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0045f3-9043-4816-bb39-301148567ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=073149702d76479f831b0ae746c18b5f)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:08.323931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0045f3-9043-4816-bb39-301148567ff6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:08.423416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0f0045f3-9043-4816-bb39-301148567ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073149702d76479f831b0ae746c18b5f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:08.432108 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0f0045f3-9043-4816-bb39-301148567ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073149702d76479f831b0ae746c18b5f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:08.433010 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0f0045f3-9043-4816-bb39-301148567ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073149702d76479f831b0ae746c18b5f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:08.449945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0f0045f3-9043-4816-bb39-301148567ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073149702d76479f831b0ae746c18b5f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:08.469133 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0f0045f3-9043-4816-bb39-301148567ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073149702d76479f831b0ae746c18b5f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:08.492048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1080/2155] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:53:08 2022] DELETE /identity/v3/users/073149702d76479f831b0ae746c18b5f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:53:09.086603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:09.089173 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:09.107312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:09.115530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebc08a-9bdf-4994-909f-ed33c90d5007 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:09.243646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:09.245496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:09.245975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:09.246540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] PATH_INFO: `/v3/projects/91e3d9befb114cc0843454480b4eb547` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:09.248642 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e3d9befb114cc0843454480b4eb547)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:09.250694 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:09.365945 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e3d9befb114cc0843454480b4eb547', 'request_id': 'req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:09.374745 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e3d9befb114cc0843454480b4eb547', 'request_id': 'req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:09.387751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e3d9befb114cc0843454480b4eb547', 'request_id': 'req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:09.432142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] The token cache is being invalidate because project 91e3d9befb114cc0843454480b4eb547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:53:09.432440 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:09.433334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:09.438323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1076/2156] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:53:09 2022] DELETE /identity/v3/projects/91e3d9befb114cc0843454480b4eb547 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:09.440310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad81b3-8c4b-4c43-b7a0-4c4e48c0eea8 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:09.441135 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad81b3-8c4b-4c43-b7a0-4c4e48c0eea8 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:09.441578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad81b3-8c4b-4c43-b7a0-4c4e48c0eea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:09.474151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:09.497504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:09.499620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:09.515640 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-19ad81b3-8c4b-4c43-b7a0-4c4e48c0eea8 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:09.517240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-19ad81b3-8c4b-4c43-b7a0-4c4e48c0eea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:09.640032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1081/2157] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:09.640795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a73ac-5159-4b26-a850-be1cdaaedbf9 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:09.649909 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c7a77-2517-437e-b092-a08e6dba2510 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:09.650495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c7a77-2517-437e-b092-a08e6dba2510 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:09.651047 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c7a77-2517-437e-b092-a08e6dba2510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:09.654366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77171-9b34-4f18-b68f-d60917446749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:09.655399 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77171-9b34-4f18-b68f-d60917446749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:09.656820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77171-9b34-4f18-b68f-d60917446749 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:09.656820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77171-9b34-4f18-b68f-d60917446749 admin admin] PATH_INFO: `/v3/users/1ee7f5d2dac94797ad7c6681b6cffcd5` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:09.661200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c77171-9b34-4f18-b68f-d60917446749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee7f5d2dac94797ad7c6681b6cffcd5)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:09.666089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c77171-9b34-4f18-b68f-d60917446749 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:09.684912 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-533c7a77-2517-437e-b092-a08e6dba2510 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:09.687216 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-533c7a77-2517-437e-b092-a08e6dba2510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:09.707436 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1082/2158] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:09.709259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c7a77-2517-437e-b092-a08e6dba2510 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:09.724870 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c7a77-2517-437e-b092-a08e6dba2510 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:09.733598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76e274-5f14-43ce-860a-efecf1362064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:09.735765 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76e274-5f14-43ce-860a-efecf1362064 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:09.736823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76e274-5f14-43ce-860a-efecf1362064 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:09.737289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76e274-5f14-43ce-860a-efecf1362064 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:09.738135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:09.739287 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76e274-5f14-43ce-860a-efecf1362064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:53:09.739875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76e274-5f14-43ce-860a-efecf1362064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:09.743443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76e274-5f14-43ce-860a-efecf1362064 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:09.762748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-91c77171-9b34-4f18-b68f-d60917446749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee7f5d2dac94797ad7c6681b6cffcd5'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:09.766951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1083/2159] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:53:09.771033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76e274-5f14-43ce-860a-efecf1362064 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:09.784389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76e274-5f14-43ce-860a-efecf1362064 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:09.789060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-91c77171-9b34-4f18-b68f-d60917446749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee7f5d2dac94797ad7c6681b6cffcd5'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:09.790443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-91c77171-9b34-4f18-b68f-d60917446749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee7f5d2dac94797ad7c6681b6cffcd5'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:09.792893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f7cd8-985d-4f31-923c-5c1a96a49d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:09.793054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-91c77171-9b34-4f18-b68f-d60917446749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee7f5d2dac94797ad7c6681b6cffcd5'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:09.793929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f7cd8-985d-4f31-923c-5c1a96a49d1d admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:09.794439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f7cd8-985d-4f31-923c-5c1a96a49d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:09.794920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f7cd8-985d-4f31-923c-5c1a96a49d1d admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:09.795838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f7cd8-985d-4f31-923c-5c1a96a49d1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:09.798167 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f7cd8-985d-4f31-923c-5c1a96a49d1d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:09.840252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-91c77171-9b34-4f18-b68f-d60917446749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee7f5d2dac94797ad7c6681b6cffcd5'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:09.879796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1077/2160] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:53:09 2022] DELETE /identity/v3/users/1ee7f5d2dac94797ad7c6681b6cffcd5 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:09.881092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1084/2161] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:53:09 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:09.882068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f7cd8-985d-4f31-923c-5c1a96a49d1d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:09.884525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9691fe-4595-4d22-b3fc-5e250c940d42 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:09.884677 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-788f7cd8-985d-4f31-923c-5c1a96a49d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:09.885268 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9691fe-4595-4d22-b3fc-5e250c940d42 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:09.885869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9691fe-4595-4d22-b3fc-5e250c940d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:09.915317 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-1d9691fe-4595-4d22-b3fc-5e250c940d42 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:09.917312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1d9691fe-4595-4d22-b3fc-5e250c940d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:09.978817 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f7cd8-985d-4f31-923c-5c1a96a49d1d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:09.985825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbd281-5499-4d78-afa4-254aa4bb8f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:09.986281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbd281-5499-4d78-afa4-254aa4bb8f60 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:09.986670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbd281-5499-4d78-afa4-254aa4bb8f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:09.987048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbd281-5499-4d78-afa4-254aa4bb8f60 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:09.987999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbd281-5499-4d78-afa4-254aa4bb8f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:09.993362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbd281-5499-4d78-afa4-254aa4bb8f60 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:10.021044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1085/2162] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:53:09 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:10.027278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbd281-5499-4d78-afa4-254aa4bb8f60 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:10.028971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1078/2163] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:10.038810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbd281-5499-4d78-afa4-254aa4bb8f60 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:10.039919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9691fe-4595-4d22-b3fc-5e250c940d42 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:10.042962 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1d9691fe-4595-4d22-b3fc-5e250c940d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:10.045941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae798e-df98-4884-921e-862f0362f60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:10.046566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae798e-df98-4884-921e-862f0362f60a admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:10.046844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae798e-df98-4884-921e-862f0362f60a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:10.047115 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae798e-df98-4884-921e-862f0362f60a admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:10.047947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae798e-df98-4884-921e-862f0362f60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:10.049606 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae798e-df98-4884-921e-862f0362f60a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:10.068042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1086/2164] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:53:10.069357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae798e-df98-4884-921e-862f0362f60a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:10.082109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae798e-df98-4884-921e-862f0362f60a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:10.089801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4068e3e-d861-4c6f-867d-361daa4da9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:10.090503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4068e3e-d861-4c6f-867d-361daa4da9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:10.090789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4068e3e-d861-4c6f-867d-361daa4da9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:10.091063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4068e3e-d861-4c6f-867d-361daa4da9a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:10.091960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4068e3e-d861-4c6f-867d-361daa4da9a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:10.093601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4068e3e-d861-4c6f-867d-361daa4da9a9 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:10.111445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1087/2165] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:53:10.116056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4068e3e-d861-4c6f-867d-361daa4da9a9 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:10.130230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4068e3e-d861-4c6f-867d-361daa4da9a9 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:10.135127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9691fe-4595-4d22-b3fc-5e250c940d42 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:10.136078 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a208d7f-1621-47e3-af09-03c183cea02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:10.137058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a208d7f-1621-47e3-af09-03c183cea02f admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:10.137058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a208d7f-1621-47e3-af09-03c183cea02f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:10.137333 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a208d7f-1621-47e3-af09-03c183cea02f admin admin] PATH_INFO: `/v3/projects/8d1bfb38d11e40c3be3d7d7dfcf5a50d/users/6923636476af49bfa366c9779f11378b/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:10.142402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509a1aa-979a-4788-af28-5638412ecbb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:10.143336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509a1aa-979a-4788-af28-5638412ecbb4 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:10.143857 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509a1aa-979a-4788-af28-5638412ecbb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:10.145012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509a1aa-979a-4788-af28-5638412ecbb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:10.160147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509a1aa-979a-4788-af28-5638412ecbb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:10.161862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a208d7f-1621-47e3-af09-03c183cea02f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1bfb38d11e40c3be3d7d7dfcf5a50d, user_id=6923636476af49bfa366c9779f11378b, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:10.163012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509a1aa-979a-4788-af28-5638412ecbb4 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:10.166665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a208d7f-1621-47e3-af09-03c183cea02f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:10.175339 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1079/2166] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:53:10.202624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1088/2167] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:53:10 2022] PUT /identity/v3/projects/8d1bfb38d11e40c3be3d7d7dfcf5a50d/users/6923636476af49bfa366c9779f11378b/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:53:10.204559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53706d45-99bb-43b4-8135-9c35dc78ae9b None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:10.205074 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53706d45-99bb-43b4-8135-9c35dc78ae9b None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:10.205562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53706d45-99bb-43b4-8135-9c35dc78ae9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:10.256753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-53706d45-99bb-43b4-8135-9c35dc78ae9b None None] MFA Rules not processed for user `6923636476af49bfa366c9779f11378b`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:10.258301 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-53706d45-99bb-43b4-8135-9c35dc78ae9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:10.348591 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:10.420234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a208d7f-1621-47e3-af09-03c183cea02f admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:10.427891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1080/2168] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:10.433724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:10.518278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a208d7f-1621-47e3-af09-03c183cea02f admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:10.527212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aadba5-4468-4bd5-a0e3-9d3e335a4adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:10.528492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aadba5-4468-4bd5-a0e3-9d3e335a4adf admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:10.528629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aadba5-4468-4bd5-a0e3-9d3e335a4adf admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:10.528882 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aadba5-4468-4bd5-a0e3-9d3e335a4adf admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:10.529531 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aadba5-4468-4bd5-a0e3-9d3e335a4adf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:10.531518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aadba5-4468-4bd5-a0e3-9d3e335a4adf admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:10.566329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53706d45-99bb-43b4-8135-9c35dc78ae9b None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:10.567860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-53706d45-99bb-43b4-8135-9c35dc78ae9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:10.593281 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53706d45-99bb-43b4-8135-9c35dc78ae9b None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:10.599471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1089/2169] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:53:10 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:10.601886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:10.602841 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:10.603390 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:10.603963 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] PATH_INFO: `/v3/projects/eac4b53b17ec466a84ab04fa38b49627` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:10.605680 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aadba5-4468-4bd5-a0e3-9d3e335a4adf admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:10.606072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eac4b53b17ec466a84ab04fa38b49627)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:10.608444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:10.686616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eac4b53b17ec466a84ab04fa38b49627', 'request_id': 'req-7408905a-d2a8-4e90-b651-70c1a086a581'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:10.694873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eac4b53b17ec466a84ab04fa38b49627', 'request_id': 'req-7408905a-d2a8-4e90-b651-70c1a086a581'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:10.696012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aadba5-4468-4bd5-a0e3-9d3e335a4adf admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:10.704918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eac4b53b17ec466a84ab04fa38b49627', 'request_id': 'req-7408905a-d2a8-4e90-b651-70c1a086a581'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:10.732298 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] The token cache is being invalidate because project eac4b53b17ec466a84ab04fa38b49627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:53:10.732778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:10.733463 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:10.737279 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1081/2170] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:53:10 2022] DELETE /identity/v3/projects/eac4b53b17ec466a84ab04fa38b49627 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:10.742710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:10.745542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:10.776494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac64523-ee03-4f04-b6a8-ac46235d98f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:10.777603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac64523-ee03-4f04-b6a8-ac46235d98f5 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:10.777888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac64523-ee03-4f04-b6a8-ac46235d98f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:10.778158 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac64523-ee03-4f04-b6a8-ac46235d98f5 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:10.779174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac64523-ee03-4f04-b6a8-ac46235d98f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:10.781148 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac64523-ee03-4f04-b6a8-ac46235d98f5 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:10.804097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1090/2171] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:53:10 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:10.808178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac64523-ee03-4f04-b6a8-ac46235d98f5 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:10.810330 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4ac64523-ee03-4f04-b6a8-ac46235d98f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:10.826378 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408905a-d2a8-4e90-b651-70c1a086a581 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:10.827752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac64523-ee03-4f04-b6a8-ac46235d98f5 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:10.832555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:10.833275 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:10.833563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:10.833841 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] PATH_INFO: `/v3/users/471a0670e24f4320b4923160bf9621bb` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:10.835157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=471a0670e24f4320b4923160bf9621bb)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:10.835302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c144ac-7ed0-412d-b8cf-15728d6d165c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:10.835950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c144ac-7ed0-412d-b8cf-15728d6d165c admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:10.836371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c144ac-7ed0-412d-b8cf-15728d6d165c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:10.836794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c144ac-7ed0-412d-b8cf-15728d6d165c admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:10.837264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:10.838217 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c144ac-7ed0-412d-b8cf-15728d6d165c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:10.841547 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c144ac-7ed0-412d-b8cf-15728d6d165c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:10.862940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1091/2172] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:53:10.867279 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c144ac-7ed0-412d-b8cf-15728d6d165c admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:10.879816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c144ac-7ed0-412d-b8cf-15728d6d165c admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:10.887594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ee190-918b-47a7-b79e-542647e00228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:10.888736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ee190-918b-47a7-b79e-542647e00228 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:10.889021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ee190-918b-47a7-b79e-542647e00228 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:10.889290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ee190-918b-47a7-b79e-542647e00228 admin admin] PATH_INFO: `/v3/projects/7a9cdafed422412f899f8bf30dc73df7/users/085e624d6c21487e8b23630be6a30991/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:10.912226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '471a0670e24f4320b4923160bf9621bb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:10.916271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ee190-918b-47a7-b79e-542647e00228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9cdafed422412f899f8bf30dc73df7, user_id=085e624d6c21487e8b23630be6a30991, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:10.921118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ee190-918b-47a7-b79e-542647e00228 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:10.923943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:10.928722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '471a0670e24f4320b4923160bf9621bb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:10.929868 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '471a0670e24f4320b4923160bf9621bb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:10.936164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '471a0670e24f4320b4923160bf9621bb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:10.951807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1092/2173] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:53:10 2022] PUT /identity/v3/projects/7a9cdafed422412f899f8bf30dc73df7/users/085e624d6c21487e8b23630be6a30991/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:53:10.952758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:10.966038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ee190-918b-47a7-b79e-542647e00228 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:10.966038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-190ee190-918b-47a7-b79e-542647e00228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:10.966481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '471a0670e24f4320b4923160bf9621bb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:10.981352 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1082/2174] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:53:10 2022] DELETE /identity/v3/users/471a0670e24f4320b4923160bf9621bb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:53:11.049070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ee190-918b-47a7-b79e-542647e00228 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:11.113187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cc7b8-4d47-4f1a-aa5e-beb0269e476e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:11.114171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cc7b8-4d47-4f1a-aa5e-beb0269e476e admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:11.114510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cc7b8-4d47-4f1a-aa5e-beb0269e476e admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:11.114775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cc7b8-4d47-4f1a-aa5e-beb0269e476e admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:11.116155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002cc7b8-4d47-4f1a-aa5e-beb0269e476e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:11.118298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002cc7b8-4d47-4f1a-aa5e-beb0269e476e admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:11.136411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1093/2175] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:53:11.138047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:11.150491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9714b-96ae-495f-b112-0699e8af8a64 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:11.155890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c823287-18ab-4588-9c07-6ffb05e942d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:11.156571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c823287-18ab-4588-9c07-6ffb05e942d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:11.157542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c823287-18ab-4588-9c07-6ffb05e942d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:11.157542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c823287-18ab-4588-9c07-6ffb05e942d8 admin admin] PATH_INFO: `/v3/projects/7a9cdafed422412f899f8bf30dc73df7/users/085e624d6c21487e8b23630be6a30991/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:11.160928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c823287-18ab-4588-9c07-6ffb05e942d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9cdafed422412f899f8bf30dc73df7, user_id=085e624d6c21487e8b23630be6a30991, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:11.161952 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c823287-18ab-4588-9c07-6ffb05e942d8 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:11.173618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1083/2176] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:53:11 2022] PUT /identity/v3/projects/7a9cdafed422412f899f8bf30dc73df7/users/085e624d6c21487e8b23630be6a30991/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:11.185166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7725b-efc0-4f9a-8e21-04ebc017746a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:11.185451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7725b-efc0-4f9a-8e21-04ebc017746a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:11.185731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7725b-efc0-4f9a-8e21-04ebc017746a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:11.220154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-95e7725b-efc0-4f9a-8e21-04ebc017746a None None] MFA Rules not processed for user `085e624d6c21487e8b23630be6a30991`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:11.221171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-95e7725b-efc0-4f9a-8e21-04ebc017746a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:11.304331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c823287-18ab-4588-9c07-6ffb05e942d8 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:11.307377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-0c823287-18ab-4588-9c07-6ffb05e942d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:11.315840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1094/2177] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:11.342120 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7725b-efc0-4f9a-8e21-04ebc017746a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:11.344018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-95e7725b-efc0-4f9a-8e21-04ebc017746a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:11.407732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c823287-18ab-4588-9c07-6ffb05e942d8 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:11.413491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:11.414140 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:11.414545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:11.414736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] PATH_INFO: `/v3/projects/7fda93032fd64612b3bfbe450524664d` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:11.416417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fda93032fd64612b3bfbe450524664d)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:11.418431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:11.430533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:11.475924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7725b-efc0-4f9a-8e21-04ebc017746a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:11.479152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fda93032fd64612b3bfbe450524664d', 'request_id': 'req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:11.485422 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fda93032fd64612b3bfbe450524664d', 'request_id': 'req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:11.496015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fda93032fd64612b3bfbe450524664d', 'request_id': 'req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:11.516320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] The token cache is being invalidate because project 7fda93032fd64612b3bfbe450524664d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:53:11.516810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:11.517352 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:11.519107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1084/2178] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:53:11 2022] DELETE /identity/v3/projects/7fda93032fd64612b3bfbe450524664d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:53:11.523731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:11.526360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:11.545382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d1f10-c2f0-4b14-9964-fb6f9ed8a8cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:11.546135 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d1f10-c2f0-4b14-9964-fb6f9ed8a8cd service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:11.546507 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d1f10-c2f0-4b14-9964-fb6f9ed8a8cd service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:11.546850 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d1f10-c2f0-4b14-9964-fb6f9ed8a8cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:11.548883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-443d1f10-c2f0-4b14-9964-fb6f9ed8a8cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:11.619851 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d93d8-0ead-4fe0-8aab-9a518fb792a6 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:11.626024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:11.626794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:11.627110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:11.627389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] PATH_INFO: `/v3/users/64f6aea849c3480fb70afc3a9e5f9529` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:11.628866 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64f6aea849c3480fb70afc3a9e5f9529)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:11.629060 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d1f10-c2f0-4b14-9964-fb6f9ed8a8cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:11.630552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d1f10-c2f0-4b14-9964-fb6f9ed8a8cd service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:11.631118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:11.641819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1095/2179] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:11.644022 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:11.695807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64f6aea849c3480fb70afc3a9e5f9529'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:11.705983 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64f6aea849c3480fb70afc3a9e5f9529'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:11.708474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64f6aea849c3480fb70afc3a9e5f9529'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:11.709350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64f6aea849c3480fb70afc3a9e5f9529'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:11.725116 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64f6aea849c3480fb70afc3a9e5f9529'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:11.743988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1085/2180] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:53:11 2022] DELETE /identity/v3/users/64f6aea849c3480fb70afc3a9e5f9529 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:53:12.161978 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d1f10-c2f0-4b14-9964-fb6f9ed8a8cd service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:12.175958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:12.178503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:12.181510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d1f10-c2f0-4b14-9964-fb6f9ed8a8cd service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:12.276765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545c4ff-fa5f-4f44-90ec-78563d38e6d3 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:12.283720 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfc6d0-02c7-42c8-96d3-f150c042f287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:12.284294 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfc6d0-02c7-42c8-96d3-f150c042f287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.284617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfc6d0-02c7-42c8-96d3-f150c042f287 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.284968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfc6d0-02c7-42c8-96d3-f150c042f287 admin admin] PATH_INFO: `/v3/users/6923636476af49bfa366c9779f11378b` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.286217 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfc6d0-02c7-42c8-96d3-f150c042f287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6923636476af49bfa366c9779f11378b)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:12.287150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26994d37-3db3-4003-b218-875287d0249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:12.288685 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26994d37-3db3-4003-b218-875287d0249f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.288685 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26994d37-3db3-4003-b218-875287d0249f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.288868 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfc6d0-02c7-42c8-96d3-f150c042f287 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:12.288945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26994d37-3db3-4003-b218-875287d0249f admin admin] PATH_INFO: `/v3/projects/6e63f8b24dc34b4aa45c45a03acb6968` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.290511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26994d37-3db3-4003-b218-875287d0249f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e63f8b24dc34b4aa45c45a03acb6968)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:12.292660 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26994d37-3db3-4003-b218-875287d0249f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:12.336196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:12.356910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-26994d37-3db3-4003-b218-875287d0249f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e63f8b24dc34b4aa45c45a03acb6968', 'request_id': 'req-26994d37-3db3-4003-b218-875287d0249f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:12.360313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c7dfc6d0-02c7-42c8-96d3-f150c042f287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6923636476af49bfa366c9779f11378b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:12.368401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-26994d37-3db3-4003-b218-875287d0249f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e63f8b24dc34b4aa45c45a03acb6968', 'request_id': 'req-26994d37-3db3-4003-b218-875287d0249f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:12.374614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c7dfc6d0-02c7-42c8-96d3-f150c042f287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6923636476af49bfa366c9779f11378b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:12.375546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c7dfc6d0-02c7-42c8-96d3-f150c042f287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6923636476af49bfa366c9779f11378b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:12.376391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c7dfc6d0-02c7-42c8-96d3-f150c042f287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6923636476af49bfa366c9779f11378b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:12.381812 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-26994d37-3db3-4003-b218-875287d0249f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e63f8b24dc34b4aa45c45a03acb6968', 'request_id': 'req-26994d37-3db3-4003-b218-875287d0249f'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:12.394303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c7dfc6d0-02c7-42c8-96d3-f150c042f287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6923636476af49bfa366c9779f11378b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:12.412561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1086/2181] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:53:12 2022] DELETE /identity/v3/users/6923636476af49bfa366c9779f11378b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:12.413185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-26994d37-3db3-4003-b218-875287d0249f admin admin] The token cache is being invalidate because project 6e63f8b24dc34b4aa45c45a03acb6968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:53:12.413644 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-26994d37-3db3-4003-b218-875287d0249f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:12.414575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-26994d37-3db3-4003-b218-875287d0249f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:12.417016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02b3c6-eb95-4a8b-abc3-ac04e1f4834a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.417390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1096/2182] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:53:12 2022] DELETE /identity/v3/projects/6e63f8b24dc34b4aa45c45a03acb6968 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:12.417560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02b3c6-eb95-4a8b-abc3-ac04e1f4834a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.418013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02b3c6-eb95-4a8b-abc3-ac04e1f4834a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.422919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bdc96-35fe-4cc0-b97c-6401d172234a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.423425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bdc96-35fe-4cc0-b97c-6401d172234a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.423879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bdc96-35fe-4cc0-b97c-6401d172234a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.427377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:12.445599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-6c02b3c6-eb95-4a8b-abc3-ac04e1f4834a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:12.448064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6c02b3c6-eb95-4a8b-abc3-ac04e1f4834a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:12.456725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-121bdc96-35fe-4cc0-b97c-6401d172234a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:12.458425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-121bdc96-35fe-4cc0-b97c-6401d172234a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:12.532042 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1087/2183] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:53:12.533938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1097/2184] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:12.543563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846917e8-f21b-4d69-a41c-778615c4af32 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.543940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846917e8-f21b-4d69-a41c-778615c4af32 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.544258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846917e8-f21b-4d69-a41c-778615c4af32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.545618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bdc96-35fe-4cc0-b97c-6401d172234a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:12.547118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-121bdc96-35fe-4cc0-b97c-6401d172234a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:12.578384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-846917e8-f21b-4d69-a41c-778615c4af32 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:12.579990 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-846917e8-f21b-4d69-a41c-778615c4af32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:12.591860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1088/2185] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:12.596255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846917e8-f21b-4d69-a41c-778615c4af32 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:12.607304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846917e8-f21b-4d69-a41c-778615c4af32 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:12.615429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e260511-5d98-4486-8289-7c0b511572a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:12.616102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e260511-5d98-4486-8289-7c0b511572a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.616382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e260511-5d98-4486-8289-7c0b511572a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.616653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e260511-5d98-4486-8289-7c0b511572a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.617325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e260511-5d98-4486-8289-7c0b511572a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:53:12.617614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e260511-5d98-4486-8289-7c0b511572a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:12.619819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e260511-5d98-4486-8289-7c0b511572a0 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:12.632519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1089/2186] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:53:12.637028 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bdc96-35fe-4cc0-b97c-6401d172234a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:12.638083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e260511-5d98-4486-8289-7c0b511572a0 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:12.641452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513e924-306b-46a7-8fe1-602be4537722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:12.642077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e924-306b-46a7-8fe1-602be4537722 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.642349 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e924-306b-46a7-8fe1-602be4537722 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.642618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e924-306b-46a7-8fe1-602be4537722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.648272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e260511-5d98-4486-8289-7c0b511572a0 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:12.652466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513e924-306b-46a7-8fe1-602be4537722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:12.653052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85f42b-bb87-4c04-bbf9-8535ee858d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:12.653662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85f42b-bb87-4c04-bbf9-8535ee858d05 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.653914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513e924-306b-46a7-8fe1-602be4537722 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:12.654039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85f42b-bb87-4c04-bbf9-8535ee858d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.654195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85f42b-bb87-4c04-bbf9-8535ee858d05 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.654822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85f42b-bb87-4c04-bbf9-8535ee858d05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:12.656842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85f42b-bb87-4c04-bbf9-8535ee858d05 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:12.668054 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1098/2187] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:12.691130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1090/2188] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:53:12 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:53:12.692815 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513e924-306b-46a7-8fe1-602be4537722 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:12.706898 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513e924-306b-46a7-8fe1-602be4537722 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:12.786229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6b3af-7b96-4ec1-aba0-3b633418862f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:12.787410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6b3af-7b96-4ec1-aba0-3b633418862f admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.787713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6b3af-7b96-4ec1-aba0-3b633418862f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.788002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6b3af-7b96-4ec1-aba0-3b633418862f admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.789116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6b3af-7b96-4ec1-aba0-3b633418862f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:12.791272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6b3af-7b96-4ec1-aba0-3b633418862f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:12.800934 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:12.822796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1099/2189] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:53:12 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:12.827287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85f42b-bb87-4c04-bbf9-8535ee858d05 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:12.839990 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85f42b-bb87-4c04-bbf9-8535ee858d05 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:12.845072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f05db-f7c3-46e9-a388-a932d5ea0c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:12.845689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f05db-f7c3-46e9-a388-a932d5ea0c3e admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.845969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f05db-f7c3-46e9-a388-a932d5ea0c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.846240 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f05db-f7c3-46e9-a388-a932d5ea0c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.846902 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f05db-f7c3-46e9-a388-a932d5ea0c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:12.848789 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f05db-f7c3-46e9-a388-a932d5ea0c3e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:12.860101 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1091/2190] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:53:12.864298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6b3af-7b96-4ec1-aba0-3b633418862f admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:12.879390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6b3af-7b96-4ec1-aba0-3b633418862f admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:12.884430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bf040-8730-4c73-80c5-68b37c3ab80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:12.885137 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bf040-8730-4c73-80c5-68b37c3ab80b admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.886166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bf040-8730-4c73-80c5-68b37c3ab80b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.886166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bf040-8730-4c73-80c5-68b37c3ab80b admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.886526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368bf040-8730-4c73-80c5-68b37c3ab80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:12.888327 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368bf040-8730-4c73-80c5-68b37c3ab80b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:12.907550 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1100/2191] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:53:12.911461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f05db-f7c3-46e9-a388-a932d5ea0c3e admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:12.922497 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f05db-f7c3-46e9-a388-a932d5ea0c3e admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:12.934317 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ba228-270d-4f7a-8740-4b0b9651fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:12.935033 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ba228-270d-4f7a-8740-4b0b9651fe5a admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.935310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ba228-270d-4f7a-8740-4b0b9651fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.935578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ba228-270d-4f7a-8740-4b0b9651fe5a admin admin] PATH_INFO: `/v3/projects/d334ad6bd6194604a61ce4ca95149993/users/ac4d918f549c41d3ae19e80cecefa95c/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.960770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ba228-270d-4f7a-8740-4b0b9651fe5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d334ad6bd6194604a61ce4ca95149993, user_id=ac4d918f549c41d3ae19e80cecefa95c, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:12.963603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ba228-270d-4f7a-8740-4b0b9651fe5a admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:12.990933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e011b5-c148-4c2a-b783-102340c118c5 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:12.991217 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e011b5-c148-4c2a-b783-102340c118c5 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:12.991490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e011b5-c148-4c2a-b783-102340c118c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:12.992501 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1092/2192] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:53:12 2022] PUT /identity/v3/projects/d334ad6bd6194604a61ce4ca95149993/users/ac4d918f549c41d3ae19e80cecefa95c/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:13.012730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-d6e011b5-c148-4c2a-b783-102340c118c5 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:13.014140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d6e011b5-c148-4c2a-b783-102340c118c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:13.033309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ba228-270d-4f7a-8740-4b0b9651fe5a admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:13.035222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-350ba228-270d-4f7a-8740-4b0b9651fe5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:13.108465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1101/2193] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:13.122298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e011b5-c148-4c2a-b783-102340c118c5 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:13.150522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ba228-270d-4f7a-8740-4b0b9651fe5a admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:13.156191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:13.156864 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:13.157148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:13.157420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] PATH_INFO: `/v3/projects/8d1bfb38d11e40c3be3d7d7dfcf5a50d` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:13.158637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d1bfb38d11e40c3be3d7d7dfcf5a50d)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:13.160634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:13.195901 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e011b5-c148-4c2a-b783-102340c118c5 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:13.200047 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bd9c7-0295-4f1b-b857-c71e7288db9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:13.200942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bd9c7-0295-4f1b-b857-c71e7288db9f service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:13.201460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bd9c7-0295-4f1b-b857-c71e7288db9f service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:13.201949 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bd9c7-0295-4f1b-b857-c71e7288db9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:13.202107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:13.215594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720bd9c7-0295-4f1b-b857-c71e7288db9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:13.217093 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720bd9c7-0295-4f1b-b857-c71e7288db9f service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:13.224124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d1bfb38d11e40c3be3d7d7dfcf5a50d', 'request_id': 'req-1b742a48-f60e-47a1-b167-c8d0f23f86b3'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:13.231258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1102/2194] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:13.243197 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d1bfb38d11e40c3be3d7d7dfcf5a50d', 'request_id': 'req-1b742a48-f60e-47a1-b167-c8d0f23f86b3'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:13.253794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d1bfb38d11e40c3be3d7d7dfcf5a50d', 'request_id': 'req-1b742a48-f60e-47a1-b167-c8d0f23f86b3'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:13.270167 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] The token cache is being invalidate because project 8d1bfb38d11e40c3be3d7d7dfcf5a50d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:53:13.270768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:13.271774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:13.273560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1093/2195] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:53:13 2022] DELETE /identity/v3/projects/8d1bfb38d11e40c3be3d7d7dfcf5a50d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:53:13.278554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bd9c7-0295-4f1b-b857-c71e7288db9f service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:13.296584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bd9c7-0295-4f1b-b857-c71e7288db9f service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:13.365686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac03a07-0379-4c36-bd40-a976c4a0e206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:13.366332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac03a07-0379-4c36-bd40-a976c4a0e206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:13.366605 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac03a07-0379-4c36-bd40-a976c4a0e206 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:13.367172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac03a07-0379-4c36-bd40-a976c4a0e206 admin admin] PATH_INFO: `/v3/users/085e624d6c21487e8b23630be6a30991` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:13.368694 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac03a07-0379-4c36-bd40-a976c4a0e206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=085e624d6c21487e8b23630be6a30991)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:13.371048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac03a07-0379-4c36-bd40-a976c4a0e206 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:13.419291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:INSERT=2,keystone:SELECT=94,keystone:DELETE=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:13.461691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-5ac03a07-0379-4c36-bd40-a976c4a0e206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '085e624d6c21487e8b23630be6a30991'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:13.470803 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-5ac03a07-0379-4c36-bd40-a976c4a0e206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '085e624d6c21487e8b23630be6a30991'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:13.471904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-5ac03a07-0379-4c36-bd40-a976c4a0e206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '085e624d6c21487e8b23630be6a30991'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:13.485667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-5ac03a07-0379-4c36-bd40-a976c4a0e206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '085e624d6c21487e8b23630be6a30991'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:13.493194 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-5ac03a07-0379-4c36-bd40-a976c4a0e206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '085e624d6c21487e8b23630be6a30991'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:13.499578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1103/2196] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:53:13 2022] DELETE /identity/v3/users/085e624d6c21487e8b23630be6a30991 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:13.965288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:13.966905 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:13.980723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b742a48-f60e-47a1-b167-c8d0f23f86b3 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:14.078592 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:14.079798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:14.080090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:14.080383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d admin admin] PATH_INFO: `/v3/projects/7a9cdafed422412f899f8bf30dc73df7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:14.081884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a9cdafed422412f899f8bf30dc73df7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:14.083833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:14.167533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a9cdafed422412f899f8bf30dc73df7', 'request_id': 'req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:14.177599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a9cdafed422412f899f8bf30dc73df7', 'request_id': 'req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:14.194903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a9cdafed422412f899f8bf30dc73df7', 'request_id': 'req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:14.224063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d admin admin] The token cache is being invalidate because project 7a9cdafed422412f899f8bf30dc73df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:53:14.224063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:14.228354 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-af4da6bd-0c3f-4e59-b8c3-6aa3eb81b11d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:14.231361 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1094/2197] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:53:13 2022] DELETE /identity/v3/projects/7a9cdafed422412f899f8bf30dc73df7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:14.236319 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65249d-73e9-40b3-a13a-1fddd592ffde None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:14.236916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65249d-73e9-40b3-a13a-1fddd592ffde None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:14.236916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65249d-73e9-40b3-a13a-1fddd592ffde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:14.265774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-2f65249d-73e9-40b3-a13a-1fddd592ffde None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:14.267046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2f65249d-73e9-40b3-a13a-1fddd592ffde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:14.386638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1104/2198] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:53:14.394379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de0a70-a041-406b-9c2e-25b2e46dbe0a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:14.394919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de0a70-a041-406b-9c2e-25b2e46dbe0a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:14.395352 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de0a70-a041-406b-9c2e-25b2e46dbe0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:14.425611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-53de0a70-a041-406b-9c2e-25b2e46dbe0a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:14.437997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-53de0a70-a041-406b-9c2e-25b2e46dbe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:14.455083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1095/2199] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:14.459923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65249d-73e9-40b3-a13a-1fddd592ffde None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:14.475448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65249d-73e9-40b3-a13a-1fddd592ffde None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:14.483408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a32b4-2619-495b-8d00-7f43e74e37b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:14.484301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a32b4-2619-495b-8d00-7f43e74e37b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:14.484824 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a32b4-2619-495b-8d00-7f43e74e37b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:14.485569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a32b4-2619-495b-8d00-7f43e74e37b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:14.486516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a32b4-2619-495b-8d00-7f43e74e37b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:53:14.487066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a32b4-2619-495b-8d00-7f43e74e37b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:14.489469 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a32b4-2619-495b-8d00-7f43e74e37b7 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:14.514552 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de0a70-a041-406b-9c2e-25b2e46dbe0a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:14.514964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1105/2200] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:53:14.526123 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de0a70-a041-406b-9c2e-25b2e46dbe0a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:14.532777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e85d44-d1f1-4b0b-83a1-a10abeb609fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:14.533586 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e85d44-d1f1-4b0b-83a1-a10abeb609fc admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:14.534044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e85d44-d1f1-4b0b-83a1-a10abeb609fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:14.534453 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e85d44-d1f1-4b0b-83a1-a10abeb609fc admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:14.535405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e85d44-d1f1-4b0b-83a1-a10abeb609fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:14.537994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e85d44-d1f1-4b0b-83a1-a10abeb609fc admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:14.577340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1096/2201] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:53:14 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:14.581581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a32b4-2619-495b-8d00-7f43e74e37b7 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:14.594747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a32b4-2619-495b-8d00-7f43e74e37b7 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:14.688822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f72be-79a5-448f-aa27-5a1e21446675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:14.689570 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f72be-79a5-448f-aa27-5a1e21446675 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:14.689956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f72be-79a5-448f-aa27-5a1e21446675 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:14.690333 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f72be-79a5-448f-aa27-5a1e21446675 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:14.691553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f72be-79a5-448f-aa27-5a1e21446675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:14.697826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f72be-79a5-448f-aa27-5a1e21446675 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:14.744655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1106/2202] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:53:14 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:14.744934 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e85d44-d1f1-4b0b-83a1-a10abeb609fc admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:14.758304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e85d44-d1f1-4b0b-83a1-a10abeb609fc admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:14.772194 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbc5e1-41db-4a0b-a16a-a9f93a3df94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:14.772821 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbc5e1-41db-4a0b-a16a-a9f93a3df94d admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:14.773106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbc5e1-41db-4a0b-a16a-a9f93a3df94d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:14.773375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbc5e1-41db-4a0b-a16a-a9f93a3df94d admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:14.774045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbc5e1-41db-4a0b-a16a-a9f93a3df94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:14.776283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbc5e1-41db-4a0b-a16a-a9f93a3df94d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:14.802633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1097/2203] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:53:14.806614 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f72be-79a5-448f-aa27-5a1e21446675 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:14.820855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f72be-79a5-448f-aa27-5a1e21446675 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:14.829881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35145e-08db-4edc-b1c1-d503ec311cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:14.830594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35145e-08db-4edc-b1c1-d503ec311cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:14.830880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35145e-08db-4edc-b1c1-d503ec311cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:14.831155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35145e-08db-4edc-b1c1-d503ec311cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:14.831856 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35145e-08db-4edc-b1c1-d503ec311cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:14.833848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35145e-08db-4edc-b1c1-d503ec311cf1 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:14.857084 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbc5e1-41db-4a0b-a16a-a9f93a3df94d admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:14.863730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1107/2204] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:53:14.873867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbc5e1-41db-4a0b-a16a-a9f93a3df94d admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:14.889657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a852d3-2cfa-4d8a-beda-12a36521cf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:14.890625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a852d3-2cfa-4d8a-beda-12a36521cf56 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:14.898324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a852d3-2cfa-4d8a-beda-12a36521cf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:14.898324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a852d3-2cfa-4d8a-beda-12a36521cf56 admin admin] PATH_INFO: `/v3/projects/8a5655a0e1a442e3aa9066de4e935f95/users/a3005cf680fc4797ade88feb094c2a46/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:14.909197 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:14.951041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a852d3-2cfa-4d8a-beda-12a36521cf56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5655a0e1a442e3aa9066de4e935f95, user_id=a3005cf680fc4797ade88feb094c2a46, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:14.951041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a852d3-2cfa-4d8a-beda-12a36521cf56 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:14.972770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1098/2205] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:53:14 2022] PUT /identity/v3/projects/8a5655a0e1a442e3aa9066de4e935f95/users/a3005cf680fc4797ade88feb094c2a46/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:14.984959 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83854a98-2371-4a88-b0e7-46a39a694113 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:14.985324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83854a98-2371-4a88-b0e7-46a39a694113 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:14.985677 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83854a98-2371-4a88-b0e7-46a39a694113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:15.008816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:15.032951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-83854a98-2371-4a88-b0e7-46a39a694113 None None] MFA Rules not processed for user `a3005cf680fc4797ade88feb094c2a46`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:15.036857 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-83854a98-2371-4a88-b0e7-46a39a694113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:15.172053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1108/2206] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:15.176141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a852d3-2cfa-4d8a-beda-12a36521cf56 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:15.256967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a852d3-2cfa-4d8a-beda-12a36521cf56 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:15.264784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a35c80-ddef-4ae5-96be-f4360f5f46e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:15.265740 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35c80-ddef-4ae5-96be-f4360f5f46e0 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:15.266231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35c80-ddef-4ae5-96be-f4360f5f46e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:15.266767 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35c80-ddef-4ae5-96be-f4360f5f46e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:15.267637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a35c80-ddef-4ae5-96be-f4360f5f46e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:15.270936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a35c80-ddef-4ae5-96be-f4360f5f46e0 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:15.309706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1099/2207] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:53:15 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:15.314296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83854a98-2371-4a88-b0e7-46a39a694113 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:15.333520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83854a98-2371-4a88-b0e7-46a39a694113 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:15.444591 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605da30-8112-4ae6-8f7f-27d94b9f5d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:15.445476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605da30-8112-4ae6-8f7f-27d94b9f5d76 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:15.446059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605da30-8112-4ae6-8f7f-27d94b9f5d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:15.446469 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605da30-8112-4ae6-8f7f-27d94b9f5d76 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:15.448163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2605da30-8112-4ae6-8f7f-27d94b9f5d76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:15.450597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2605da30-8112-4ae6-8f7f-27d94b9f5d76 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:15.486230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1109/2208] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:53:15 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:15.490141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a35c80-ddef-4ae5-96be-f4360f5f46e0 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:15.503737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a35c80-ddef-4ae5-96be-f4360f5f46e0 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:15.512992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aff503-3853-4a3d-bb07-dc72fb17fbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:15.514026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aff503-3853-4a3d-bb07-dc72fb17fbc8 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:15.514579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aff503-3853-4a3d-bb07-dc72fb17fbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:15.515385 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aff503-3853-4a3d-bb07-dc72fb17fbc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:15.517208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aff503-3853-4a3d-bb07-dc72fb17fbc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:15.519900 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aff503-3853-4a3d-bb07-dc72fb17fbc8 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:15.539137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1100/2209] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:53:15.540286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605da30-8112-4ae6-8f7f-27d94b9f5d76 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:15.562564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605da30-8112-4ae6-8f7f-27d94b9f5d76 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:15.570575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc956a-945b-4488-8f1f-8051d625ee03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:15.571536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc956a-945b-4488-8f1f-8051d625ee03 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:15.572678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc956a-945b-4488-8f1f-8051d625ee03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:15.573168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc956a-945b-4488-8f1f-8051d625ee03 admin admin] PATH_INFO: `/v3/projects/930d5fcc48124368ba2b16d3bbaeece8/users/00632e305e9e4c8894025667851e15c1/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:15.607382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc956a-945b-4488-8f1f-8051d625ee03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=930d5fcc48124368ba2b16d3bbaeece8, user_id=00632e305e9e4c8894025667851e15c1, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:15.612420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc956a-945b-4488-8f1f-8051d625ee03 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:15.659559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1110/2210] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:53:15 2022] PUT /identity/v3/projects/930d5fcc48124368ba2b16d3bbaeece8/users/00632e305e9e4c8894025667851e15c1/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:15.663501 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aff503-3853-4a3d-bb07-dc72fb17fbc8 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:15.678486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aff503-3853-4a3d-bb07-dc72fb17fbc8 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:15.850884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce2a28-42d3-4dad-a602-07b22ba7c28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:15.852899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce2a28-42d3-4dad-a602-07b22ba7c28b admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:15.855617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce2a28-42d3-4dad-a602-07b22ba7c28b admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:15.856578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce2a28-42d3-4dad-a602-07b22ba7c28b admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:15.858098 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce2a28-42d3-4dad-a602-07b22ba7c28b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:15.863771 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce2a28-42d3-4dad-a602-07b22ba7c28b admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:15.896922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1101/2211] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:53:15.900259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc956a-945b-4488-8f1f-8051d625ee03 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:15.912650 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc956a-945b-4488-8f1f-8051d625ee03 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:15.919296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32a167-7862-4eea-a0dc-ffb0afc17818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:15.920116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32a167-7862-4eea-a0dc-ffb0afc17818 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:15.920508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32a167-7862-4eea-a0dc-ffb0afc17818 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:15.920793 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32a167-7862-4eea-a0dc-ffb0afc17818 admin admin] PATH_INFO: `/v3/projects/930d5fcc48124368ba2b16d3bbaeece8/users/00632e305e9e4c8894025667851e15c1/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:15.929955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32a167-7862-4eea-a0dc-ffb0afc17818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=930d5fcc48124368ba2b16d3bbaeece8, user_id=00632e305e9e4c8894025667851e15c1, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:15.935784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32a167-7862-4eea-a0dc-ffb0afc17818 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:15.963997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1111/2212] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:53:15 2022] PUT /identity/v3/projects/930d5fcc48124368ba2b16d3bbaeece8/users/00632e305e9e4c8894025667851e15c1/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:15.969286 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a88fa-b043-43cc-92fe-3a90e4d938aa None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:15.969629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a88fa-b043-43cc-92fe-3a90e4d938aa None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:15.969848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a88fa-b043-43cc-92fe-3a90e4d938aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:16.020234 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-962a88fa-b043-43cc-92fe-3a90e4d938aa None None] MFA Rules not processed for user `00632e305e9e4c8894025667851e15c1`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:16.021676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-962a88fa-b043-43cc-92fe-3a90e4d938aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:16.121211 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:16.175745 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1102/2213] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:16.187785 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32a167-7862-4eea-a0dc-ffb0afc17818 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:16.189879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2e32a167-7862-4eea-a0dc-ffb0afc17818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:16.371956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32a167-7862-4eea-a0dc-ffb0afc17818 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:16.388241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f81b07-b7c3-45d5-b2a3-7b5c3f6b4486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:16.389079 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f81b07-b7c3-45d5-b2a3-7b5c3f6b4486 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:16.389492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f81b07-b7c3-45d5-b2a3-7b5c3f6b4486 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:16.403927 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f81b07-b7c3-45d5-b2a3-7b5c3f6b4486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:16.411026 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:16.506971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f81b07-b7c3-45d5-b2a3-7b5c3f6b4486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:16.509078 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f81b07-b7c3-45d5-b2a3-7b5c3f6b4486 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:16.522216 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1112/2214] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:19.535926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a88fa-b043-43cc-92fe-3a90e4d938aa None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:19.628870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a88fa-b043-43cc-92fe-3a90e4d938aa None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:19.642306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8636c-3872-44b4-9ba5-be4099fbac67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:19.642953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8636c-3872-44b4-9ba5-be4099fbac67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:19.643235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8636c-3872-44b4-9ba5-be4099fbac67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:19.643506 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8636c-3872-44b4-9ba5-be4099fbac67 admin admin] PATH_INFO: `/v3/users/a3005cf680fc4797ade88feb094c2a46` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:19.648831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8636c-3872-44b4-9ba5-be4099fbac67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3005cf680fc4797ade88feb094c2a46)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:19.654842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8636c-3872-44b4-9ba5-be4099fbac67 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:19.720748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b2bbb-b613-46ce-8ca1-c4d712fedc7a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:19.722772 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b2bbb-b613-46ce-8ca1-c4d712fedc7a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:19.723261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b2bbb-b613-46ce-8ca1-c4d712fedc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:19.759993 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-964b2bbb-b613-46ce-8ca1-c4d712fedc7a None None] MFA Rules not processed for user `ac4d918f549c41d3ae19e80cecefa95c`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:19.762037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-964b2bbb-b613-46ce-8ca1-c4d712fedc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:19.765535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-48f8636c-3872-44b4-9ba5-be4099fbac67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3005cf680fc4797ade88feb094c2a46'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:19.783317 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-48f8636c-3872-44b4-9ba5-be4099fbac67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3005cf680fc4797ade88feb094c2a46'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:19.784764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-48f8636c-3872-44b4-9ba5-be4099fbac67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3005cf680fc4797ade88feb094c2a46'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:19.785884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-48f8636c-3872-44b4-9ba5-be4099fbac67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3005cf680fc4797ade88feb094c2a46'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:19.811364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-48f8636c-3872-44b4-9ba5-be4099fbac67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3005cf680fc4797ade88feb094c2a46'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:19.845841 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1103/2215] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:53:19 2022] DELETE /identity/v3/users/a3005cf680fc4797ade88feb094c2a46 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:19.850679 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766468a-ede0-4b66-877f-d05dcd09a21f None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:19.850968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766468a-ede0-4b66-877f-d05dcd09a21f None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:19.851246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766468a-ede0-4b66-877f-d05dcd09a21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:19.885282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-1766468a-ede0-4b66-877f-d05dcd09a21f None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:19.889581 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1766468a-ede0-4b66-877f-d05dcd09a21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:19.986810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1113/2216] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:53:20.002273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b2bbb-b613-46ce-8ca1-c4d712fedc7a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:20.014771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-964b2bbb-b613-46ce-8ca1-c4d712fedc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:20.038469 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b2bbb-b613-46ce-8ca1-c4d712fedc7a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:20.064875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:20.079923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1104/2217] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:20.080800 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766468a-ede0-4b66-877f-d05dcd09a21f None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:20.082179 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1766468a-ede0-4b66-877f-d05dcd09a21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:20.101417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766468a-ede0-4b66-877f-d05dcd09a21f None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:20.117425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126c43e-10b4-4460-864a-e868e132954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:20.118054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126c43e-10b4-4460-864a-e868e132954d admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:20.118330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126c43e-10b4-4460-864a-e868e132954d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:20.119302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126c43e-10b4-4460-864a-e868e132954d admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:20.119969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126c43e-10b4-4460-864a-e868e132954d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:20.121887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126c43e-10b4-4460-864a-e868e132954d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:20.135243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:20.136040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:20.136401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:20.136674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] PATH_INFO: `/v3/users/91e84988e6284c3a9760396c79e2269d` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:20.137921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91e84988e6284c3a9760396c79e2269d)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:20.139969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:20.155003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1105/2218] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:53:20 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:20.157113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a74b50-57b1-4fb1-935d-d8da910cd71f None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:20.157473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a74b50-57b1-4fb1-935d-d8da910cd71f None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:20.157814 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a74b50-57b1-4fb1-935d-d8da910cd71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:20.180624 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-56a74b50-57b1-4fb1-935d-d8da910cd71f None None] MFA Rules not processed for user `ae0dfb7613e1432dacbfc6011efaa0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:20.182782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-56a74b50-57b1-4fb1-935d-d8da910cd71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:20.194166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:20.210152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91e84988e6284c3a9760396c79e2269d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:20.219270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91e84988e6284c3a9760396c79e2269d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:20.220184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91e84988e6284c3a9760396c79e2269d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:20.238275 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91e84988e6284c3a9760396c79e2269d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:20.246918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1106/2219] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:20.248562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91e84988e6284c3a9760396c79e2269d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:20.252198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a74b50-57b1-4fb1-935d-d8da910cd71f None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:20.253665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-56a74b50-57b1-4fb1-935d-d8da910cd71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:20.256466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1114/2220] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:53:19 2022] DELETE /identity/v3/users/91e84988e6284c3a9760396c79e2269d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:20.261185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:20.262597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:20.336822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a74b50-57b1-4fb1-935d-d8da910cd71f None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:20.357679 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20deb6c4-5a82-46be-9cfd-967f748d9139 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:20.363248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d67ad2-4cd1-42a8-b942-0400638fa6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:20.364297 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d67ad2-4cd1-42a8-b942-0400638fa6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:20.364844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d67ad2-4cd1-42a8-b942-0400638fa6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:20.365285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d67ad2-4cd1-42a8-b942-0400638fa6d2 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:20.366956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d67ad2-4cd1-42a8-b942-0400638fa6d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:20.371058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d67ad2-4cd1-42a8-b942-0400638fa6d2 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:20.399020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1115/2221] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:53:20 2022] POST /identity/v3/users => generated 296 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:20.400509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd949a3-6b01-4a9f-8c16-ca589583f8a1 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:20.400804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd949a3-6b01-4a9f-8c16-ca589583f8a1 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:20.401086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd949a3-6b01-4a9f-8c16-ca589583f8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:20.424536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea067fe-10b7-482d-99f7-d08e3784c898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:20.425182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea067fe-10b7-482d-99f7-d08e3784c898 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:20.425460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea067fe-10b7-482d-99f7-d08e3784c898 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:20.425731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea067fe-10b7-482d-99f7-d08e3784c898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:20.428019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3ea067fe-10b7-482d-99f7-d08e3784c898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:20.437493 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-cfd949a3-6b01-4a9f-8c16-ca589583f8a1 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:20.438738 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-cfd949a3-6b01-4a9f-8c16-ca589583f8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:20.451944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1116/2222] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:20.453116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd949a3-6b01-4a9f-8c16-ca589583f8a1 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:20.464367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea067fe-10b7-482d-99f7-d08e3784c898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:20.466597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd949a3-6b01-4a9f-8c16-ca589583f8a1 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:20.466954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea067fe-10b7-482d-99f7-d08e3784c898 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:20.471611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad6e44-9cd0-48ba-8746-7202536e706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:20.472281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad6e44-9cd0-48ba-8746-7202536e706c admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:20.472683 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad6e44-9cd0-48ba-8746-7202536e706c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:20.472905 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad6e44-9cd0-48ba-8746-7202536e706c admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:20.473566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad6e44-9cd0-48ba-8746-7202536e706c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:20.475271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad6e44-9cd0-48ba-8746-7202536e706c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:20.490407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1107/2223] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:20.491394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1117/2224] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:53:20.492887 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad6e44-9cd0-48ba-8746-7202536e706c admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:20.496965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea067fe-10b7-482d-99f7-d08e3784c898 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:20.500952 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3ea067fe-10b7-482d-99f7-d08e3784c898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:20.505877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad6e44-9cd0-48ba-8746-7202536e706c admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:20.519018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae6a2a-75ca-4a7e-af7e-38d317895803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:20.519018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae6a2a-75ca-4a7e-af7e-38d317895803 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:20.519273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae6a2a-75ca-4a7e-af7e-38d317895803 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:20.520223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae6a2a-75ca-4a7e-af7e-38d317895803 admin admin] PATH_INFO: `/v3/projects/8b91274d490e44c09edc6af01a0e13fb/users/8ef862d6f6674114b783993006f68258/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:20.528027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea067fe-10b7-482d-99f7-d08e3784c898 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:20.554918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4059b5-b89f-4394-8908-558dcb6e5527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:20.556738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4059b5-b89f-4394-8908-558dcb6e5527 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:20.556738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4059b5-b89f-4394-8908-558dcb6e5527 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:20.557280 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4059b5-b89f-4394-8908-558dcb6e5527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:20.587457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4059b5-b89f-4394-8908-558dcb6e5527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:20.588975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae6a2a-75ca-4a7e-af7e-38d317895803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b91274d490e44c09edc6af01a0e13fb, user_id=8ef862d6f6674114b783993006f68258, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:20.589214 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4059b5-b89f-4394-8908-558dcb6e5527 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:20.592009 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae6a2a-75ca-4a7e-af7e-38d317895803 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:20.608395 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1108/2225] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:20.611731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1118/2226] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:53:20 2022] PUT /identity/v3/projects/8b91274d490e44c09edc6af01a0e13fb/users/8ef862d6f6674114b783993006f68258/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:20.615985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4059b5-b89f-4394-8908-558dcb6e5527 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:20.643208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4059b5-b89f-4394-8908-558dcb6e5527 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:20.686317 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:20.735712 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7938c02-23e2-4b34-827f-4c6877f5d621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:20.736504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7938c02-23e2-4b34-827f-4c6877f5d621 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:20.736805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7938c02-23e2-4b34-827f-4c6877f5d621 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:20.737183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7938c02-23e2-4b34-827f-4c6877f5d621 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:20.738109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7938c02-23e2-4b34-827f-4c6877f5d621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:20.740286 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7938c02-23e2-4b34-827f-4c6877f5d621 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:20.756057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1109/2227] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:53:20.760209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae6a2a-75ca-4a7e-af7e-38d317895803 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:20.773371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae6a2a-75ca-4a7e-af7e-38d317895803 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:20.778688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d2f41-b189-48f3-b233-882a60e264c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:20.779357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d2f41-b189-48f3-b233-882a60e264c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:20.779635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d2f41-b189-48f3-b233-882a60e264c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:20.780412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d2f41-b189-48f3-b233-882a60e264c4 admin admin] PATH_INFO: `/v3/projects/8b91274d490e44c09edc6af01a0e13fb/users/8ef862d6f6674114b783993006f68258/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:20.783559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d2f41-b189-48f3-b233-882a60e264c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b91274d490e44c09edc6af01a0e13fb, user_id=8ef862d6f6674114b783993006f68258, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:20.786213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d2f41-b189-48f3-b233-882a60e264c4 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:20.800764 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1119/2228] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:53:20 2022] PUT /identity/v3/projects/8b91274d490e44c09edc6af01a0e13fb/users/8ef862d6f6674114b783993006f68258/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:53:21.305173 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7938c02-23e2-4b34-827f-4c6877f5d621 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:21.308463 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b7938c02-23e2-4b34-827f-4c6877f5d621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:21.360305 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7938c02-23e2-4b34-827f-4c6877f5d621 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:21.607329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:21.608408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:21.608408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:21.608739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] PATH_INFO: `/v3/projects/8a5655a0e1a442e3aa9066de4e935f95` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:21.609978 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a5655a0e1a442e3aa9066de4e935f95)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:21.612326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:21.666441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d2f41-b189-48f3-b233-882a60e264c4 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:21.668321 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ec7d2f41-b189-48f3-b233-882a60e264c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:21.697090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d2f41-b189-48f3-b233-882a60e264c4 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:21.730799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a5655a0e1a442e3aa9066de4e935f95', 'request_id': 'req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:21.757140 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a5655a0e1a442e3aa9066de4e935f95', 'request_id': 'req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:21.776472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a5655a0e1a442e3aa9066de4e935f95', 'request_id': 'req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:21.806663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] The token cache is being invalidate because project 8a5655a0e1a442e3aa9066de4e935f95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:53:21.806970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:21.809338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:21.813276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1110/2229] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:53:21 2022] DELETE /identity/v3/projects/8a5655a0e1a442e3aa9066de4e935f95 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:21.820886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:21.821342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:21.823433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:21.858012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72312bed-cd67-4e0e-babf-f793ea74c35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:21.859015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72312bed-cd67-4e0e-babf-f793ea74c35a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:21.859491 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72312bed-cd67-4e0e-babf-f793ea74c35a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:21.859980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72312bed-cd67-4e0e-babf-f793ea74c35a admin admin] PATH_INFO: `/v3/projects/a2c6091e7a344c4f9e53cc1544466daf` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:21.861390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72312bed-cd67-4e0e-babf-f793ea74c35a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2c6091e7a344c4f9e53cc1544466daf)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:21.863528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72312bed-cd67-4e0e-babf-f793ea74c35a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:21.961264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:21.964667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-72312bed-cd67-4e0e-babf-f793ea74c35a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2c6091e7a344c4f9e53cc1544466daf', 'request_id': 'req-72312bed-cd67-4e0e-babf-f793ea74c35a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:21.971443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d067d-787d-4f5d-8ef2-e16996c2f62c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:21.980529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-72312bed-cd67-4e0e-babf-f793ea74c35a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2c6091e7a344c4f9e53cc1544466daf', 'request_id': 'req-72312bed-cd67-4e0e-babf-f793ea74c35a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:21.998839 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-72312bed-cd67-4e0e-babf-f793ea74c35a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2c6091e7a344c4f9e53cc1544466daf', 'request_id': 'req-72312bed-cd67-4e0e-babf-f793ea74c35a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:22.016787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-72312bed-cd67-4e0e-babf-f793ea74c35a admin admin] The token cache is being invalidate because project a2c6091e7a344c4f9e53cc1544466daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:53:22.017322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-72312bed-cd67-4e0e-babf-f793ea74c35a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:22.019841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-72312bed-cd67-4e0e-babf-f793ea74c35a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:22.032166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1120/2230] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:53:21 2022] DELETE /identity/v3/projects/a2c6091e7a344c4f9e53cc1544466daf => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:53:22.036777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50244bb-4d72-499e-bdf6-eebf45af0e7a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:22.037256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50244bb-4d72-499e-bdf6-eebf45af0e7a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:22.037803 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50244bb-4d72-499e-bdf6-eebf45af0e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:22.066991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-e50244bb-4d72-499e-bdf6-eebf45af0e7a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:22.068790 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e50244bb-4d72-499e-bdf6-eebf45af0e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:22.089773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:22.090589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:22.090994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:22.091639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] PATH_INFO: `/v3/users/00632e305e9e4c8894025667851e15c1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:22.094071 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00632e305e9e4c8894025667851e15c1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:22.096656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:22.193864 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00632e305e9e4c8894025667851e15c1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:22.212955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00632e305e9e4c8894025667851e15c1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:22.214947 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00632e305e9e4c8894025667851e15c1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:22.216302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00632e305e9e4c8894025667851e15c1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:22.220776 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1121/2231] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:22.229474 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bb59-7c4f-4590-8e0c-66f05b0125d1 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:22.231326 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bb59-7c4f-4590-8e0c-66f05b0125d1 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:22.231917 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bb59-7c4f-4590-8e0c-66f05b0125d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:22.236792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00632e305e9e4c8894025667851e15c1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:22.266896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-6db0bb59-7c4f-4590-8e0c-66f05b0125d1 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:22.268298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6db0bb59-7c4f-4590-8e0c-66f05b0125d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:22.268954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1111/2232] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:53:21 2022] DELETE /identity/v3/users/00632e305e9e4c8894025667851e15c1 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:22.415573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1122/2233] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:22.416885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:22.429900 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc2b6f-e84c-4062-810d-78bfb3273a9f admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:22.438778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626a423-9e27-432b-b6cb-7972bbcc3707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:22.440357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626a423-9e27-432b-b6cb-7972bbcc3707 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:22.440636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626a423-9e27-432b-b6cb-7972bbcc3707 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:22.443457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626a423-9e27-432b-b6cb-7972bbcc3707 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:22.446149 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626a423-9e27-432b-b6cb-7972bbcc3707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:53:22.448044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626a423-9e27-432b-b6cb-7972bbcc3707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:22.452838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626a423-9e27-432b-b6cb-7972bbcc3707 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:22.475725 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1112/2234] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:53:22.478260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bb59-7c4f-4590-8e0c-66f05b0125d1 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:22.492372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bb59-7c4f-4590-8e0c-66f05b0125d1 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:22.508513 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e973f66-182d-4b45-9146-a8a981fdaf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:22.510722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e973f66-182d-4b45-9146-a8a981fdaf0d admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:22.511477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e973f66-182d-4b45-9146-a8a981fdaf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:22.511971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e973f66-182d-4b45-9146-a8a981fdaf0d admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:22.517619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e973f66-182d-4b45-9146-a8a981fdaf0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:22.520354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e973f66-182d-4b45-9146-a8a981fdaf0d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:22.565967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:22.585751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626a423-9e27-432b-b6cb-7972bbcc3707 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:22.599790 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1123/2235] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:53:22 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:22.607111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626a423-9e27-432b-b6cb-7972bbcc3707 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:22.753860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69c7e5-688d-4437-bae8-2465ee7a7ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:22.756475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69c7e5-688d-4437-bae8-2465ee7a7ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:22.756851 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69c7e5-688d-4437-bae8-2465ee7a7ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:22.757176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69c7e5-688d-4437-bae8-2465ee7a7ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:22.761407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69c7e5-688d-4437-bae8-2465ee7a7ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:22.763539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69c7e5-688d-4437-bae8-2465ee7a7ae9 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:22.785039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:22.805132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1113/2236] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:53:22 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:53:22.808396 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e973f66-182d-4b45-9146-a8a981fdaf0d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:22.828911 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e973f66-182d-4b45-9146-a8a981fdaf0d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:22.835801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135ec64-1758-47df-a778-8c55712195e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:22.836485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135ec64-1758-47df-a778-8c55712195e7 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:22.836863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135ec64-1758-47df-a778-8c55712195e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:22.837367 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135ec64-1758-47df-a778-8c55712195e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:22.838144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135ec64-1758-47df-a778-8c55712195e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:22.840318 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135ec64-1758-47df-a778-8c55712195e7 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:22.860960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1124/2237] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:53:22.863007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69c7e5-688d-4437-bae8-2465ee7a7ae9 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:22.881526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69c7e5-688d-4437-bae8-2465ee7a7ae9 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:22.910824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993baf38-6cb3-458a-8764-b2a5f4c75e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:22.911516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993baf38-6cb3-458a-8764-b2a5f4c75e05 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:22.911761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993baf38-6cb3-458a-8764-b2a5f4c75e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:22.912083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993baf38-6cb3-458a-8764-b2a5f4c75e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:22.912742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993baf38-6cb3-458a-8764-b2a5f4c75e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:22.914548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993baf38-6cb3-458a-8764-b2a5f4c75e05 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:22.946550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1114/2238] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:53:22.950791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135ec64-1758-47df-a778-8c55712195e7 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:22.964524 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135ec64-1758-47df-a778-8c55712195e7 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:22.970341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577f320-514a-43e3-ac11-97ff91f88c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:22.972271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577f320-514a-43e3-ac11-97ff91f88c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:22.972653 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577f320-514a-43e3-ac11-97ff91f88c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:22.972938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577f320-514a-43e3-ac11-97ff91f88c0a admin admin] PATH_INFO: `/v3/projects/49b5f78066ec41e98faa23bacd4881cf/users/2ca1c6464b6d432ab1bd3898c69a3e47/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:22.999732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577f320-514a-43e3-ac11-97ff91f88c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b5f78066ec41e98faa23bacd4881cf, user_id=2ca1c6464b6d432ab1bd3898c69a3e47, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:23.007164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577f320-514a-43e3-ac11-97ff91f88c0a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:23.025544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993baf38-6cb3-458a-8764-b2a5f4c75e05 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:23.028004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-993baf38-6cb3-458a-8764-b2a5f4c75e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:23.050403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1125/2239] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:53:22 2022] PUT /identity/v3/projects/49b5f78066ec41e98faa23bacd4881cf/users/2ca1c6464b6d432ab1bd3898c69a3e47/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:23.051935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c374c-1bf3-45f8-ab2c-e3169bfae6f1 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:23.052240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c374c-1bf3-45f8-ab2c-e3169bfae6f1 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:23.052687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c374c-1bf3-45f8-ab2c-e3169bfae6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:23.053252 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993baf38-6cb3-458a-8764-b2a5f4c75e05 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:23.105764 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-7e4c374c-1bf3-45f8-ab2c-e3169bfae6f1 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:23.107534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7e4c374c-1bf3-45f8-ab2c-e3169bfae6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:23.183541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535dc85-f335-44bb-956a-d7476a9783b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:23.184753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535dc85-f335-44bb-956a-d7476a9783b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:23.185144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535dc85-f335-44bb-956a-d7476a9783b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:23.185675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535dc85-f335-44bb-956a-d7476a9783b0 admin admin] PATH_INFO: `/v3/projects/930d5fcc48124368ba2b16d3bbaeece8` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:23.187427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535dc85-f335-44bb-956a-d7476a9783b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=930d5fcc48124368ba2b16d3bbaeece8)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:23.190505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535dc85-f335-44bb-956a-d7476a9783b0 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:23.223831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1126/2240] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:23.235188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c374c-1bf3-45f8-ab2c-e3169bfae6f1 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:23.238619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7e4c374c-1bf3-45f8-ab2c-e3169bfae6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:23.282487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0535dc85-f335-44bb-956a-d7476a9783b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '930d5fcc48124368ba2b16d3bbaeece8', 'request_id': 'req-0535dc85-f335-44bb-956a-d7476a9783b0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:23.289965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0535dc85-f335-44bb-956a-d7476a9783b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '930d5fcc48124368ba2b16d3bbaeece8', 'request_id': 'req-0535dc85-f335-44bb-956a-d7476a9783b0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:23.300663 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0535dc85-f335-44bb-956a-d7476a9783b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '930d5fcc48124368ba2b16d3bbaeece8', 'request_id': 'req-0535dc85-f335-44bb-956a-d7476a9783b0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:23.316869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0535dc85-f335-44bb-956a-d7476a9783b0 admin admin] The token cache is being invalidate because project 930d5fcc48124368ba2b16d3bbaeece8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:53:23.317417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0535dc85-f335-44bb-956a-d7476a9783b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:23.318627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0535dc85-f335-44bb-956a-d7476a9783b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:53:23.320929 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1115/2241] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:53:23 2022] DELETE /identity/v3/projects/930d5fcc48124368ba2b16d3bbaeece8 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:23.328338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bf0c3-92f5-4e19-8ba7-ce5a7cfd6abb None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:23.328338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bf0c3-92f5-4e19-8ba7-ce5a7cfd6abb None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:23.328338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bf0c3-92f5-4e19-8ba7-ce5a7cfd6abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:23.358619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c374c-1bf3-45f8-ab2c-e3169bfae6f1 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:23.375268 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a27bf0c3-92f5-4e19-8ba7-ce5a7cfd6abb None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:23.376589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a27bf0c3-92f5-4e19-8ba7-ce5a7cfd6abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:23.450987 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956a313-0424-454b-b6c0-c7ad60f9f26d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:23.451628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956a313-0424-454b-b6c0-c7ad60f9f26d service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:23.451933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956a313-0424-454b-b6c0-c7ad60f9f26d service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:23.452628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956a313-0424-454b-b6c0-c7ad60f9f26d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:23.457779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a956a313-0424-454b-b6c0-c7ad60f9f26d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:23.525506 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1116/2242] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:53:23.526806 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601091b3-9e29-434b-bd93-78d87a28aa8c None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:23.527750 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601091b3-9e29-434b-bd93-78d87a28aa8c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:23.528037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601091b3-9e29-434b-bd93-78d87a28aa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:23.552234 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-601091b3-9e29-434b-bd93-78d87a28aa8c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:23.553004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:23.553818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-601091b3-9e29-434b-bd93-78d87a28aa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:23.584333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1117/2243] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:23.589342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601091b3-9e29-434b-bd93-78d87a28aa8c None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:23.611454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601091b3-9e29-434b-bd93-78d87a28aa8c None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:23.625978 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956a313-0424-454b-b6c0-c7ad60f9f26d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:23.628914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956a313-0424-454b-b6c0-c7ad60f9f26d service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:23.633591 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb41456-019a-4f8b-aa25-bcdd63c13141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:23.634866 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb41456-019a-4f8b-aa25-bcdd63c13141 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:23.635150 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb41456-019a-4f8b-aa25-bcdd63c13141 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:23.635425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb41456-019a-4f8b-aa25-bcdd63c13141 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:23.636143 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb41456-019a-4f8b-aa25-bcdd63c13141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:53:23.636455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb41456-019a-4f8b-aa25-bcdd63c13141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:23.638092 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb41456-019a-4f8b-aa25-bcdd63c13141 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:23.645252 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1127/2244] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:53:23.712370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:23.735721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1118/2245] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:53:23.738583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956a313-0424-454b-b6c0-c7ad60f9f26d service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:23.760794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956a313-0424-454b-b6c0-c7ad60f9f26d service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:23.776683 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf0c76-07e2-42bc-84c1-0f165327035b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:23.777325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf0c76-07e2-42bc-84c1-0f165327035b admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:23.777607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf0c76-07e2-42bc-84c1-0f165327035b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:23.777885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf0c76-07e2-42bc-84c1-0f165327035b admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:23.782503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf0c76-07e2-42bc-84c1-0f165327035b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:23.791862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf0c76-07e2-42bc-84c1-0f165327035b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:23.904012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1128/2246] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:53:23 2022] POST /identity/v3/projects => generated 369 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:23.908333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb41456-019a-4f8b-aa25-bcdd63c13141 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:23.933267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb41456-019a-4f8b-aa25-bcdd63c13141 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:24.171444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721b85a-4d4e-4d1e-859c-b510c48ccc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:24.182825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721b85a-4d4e-4d1e-859c-b510c48ccc9e admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:24.183402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721b85a-4d4e-4d1e-859c-b510c48ccc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:24.183847 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721b85a-4d4e-4d1e-859c-b510c48ccc9e admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:24.184546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721b85a-4d4e-4d1e-859c-b510c48ccc9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:24.186566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721b85a-4d4e-4d1e-859c-b510c48ccc9e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:24.260656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1119/2247] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:53:23 2022] POST /identity/v3/users => generated 285 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:53:24.262030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf0c76-07e2-42bc-84c1-0f165327035b admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:24.278441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf0c76-07e2-42bc-84c1-0f165327035b admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:24.302353 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1292c5a-2a0a-45fa-8cac-a05445ec95dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:24.309023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1292c5a-2a0a-45fa-8cac-a05445ec95dc admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:24.309915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1292c5a-2a0a-45fa-8cac-a05445ec95dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:24.312300 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1292c5a-2a0a-45fa-8cac-a05445ec95dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:24.336673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1292c5a-2a0a-45fa-8cac-a05445ec95dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:24.344198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1292c5a-2a0a-45fa-8cac-a05445ec95dc admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:24.394089 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1129/2248] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:53:24.398834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721b85a-4d4e-4d1e-859c-b510c48ccc9e admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:24.419444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721b85a-4d4e-4d1e-859c-b510c48ccc9e admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:24.428372 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443267a-aec0-49ce-9b65-5cf35017676d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:24.433924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443267a-aec0-49ce-9b65-5cf35017676d admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:24.435562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443267a-aec0-49ce-9b65-5cf35017676d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:24.436669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443267a-aec0-49ce-9b65-5cf35017676d admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:24.439498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443267a-aec0-49ce-9b65-5cf35017676d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:24.458587 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443267a-aec0-49ce-9b65-5cf35017676d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:24.477229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1120/2249] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:53:24.481893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1292c5a-2a0a-45fa-8cac-a05445ec95dc admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:24.493929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1292c5a-2a0a-45fa-8cac-a05445ec95dc admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:24.502085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4a1b4-69b1-40fd-882e-ead146f81f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:24.503271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4a1b4-69b1-40fd-882e-ead146f81f7f admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:24.503799 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4a1b4-69b1-40fd-882e-ead146f81f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:24.504765 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4a1b4-69b1-40fd-882e-ead146f81f7f admin admin] PATH_INFO: `/v3/projects/49606056b6544cbabc16d65cb30b4c70/users/757c68d99b4c4bf9bb872070f6d83db6/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:24.552920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4a1b4-69b1-40fd-882e-ead146f81f7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=49606056b6544cbabc16d65cb30b4c70, user_id=757c68d99b4c4bf9bb872070f6d83db6, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:24.558462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4a1b4-69b1-40fd-882e-ead146f81f7f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:24.612360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1130/2250] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:53:24 2022] PUT /identity/v3/projects/49606056b6544cbabc16d65cb30b4c70/users/757c68d99b4c4bf9bb872070f6d83db6/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:53:24.618451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773064d5-b587-4396-acb1-fe22e292ab0e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:24.618826 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773064d5-b587-4396-acb1-fe22e292ab0e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:24.619174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773064d5-b587-4396-acb1-fe22e292ab0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:24.754894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-773064d5-b587-4396-acb1-fe22e292ab0e None None] MFA Rules not processed for user `757c68d99b4c4bf9bb872070f6d83db6`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:24.756927 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-773064d5-b587-4396-acb1-fe22e292ab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:24.947735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1121/2251] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:24.959957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4a1b4-69b1-40fd-882e-ead146f81f7f admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:24.962018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d3a4a1b4-69b1-40fd-882e-ead146f81f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:25.291875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4a1b4-69b1-40fd-882e-ead146f81f7f admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:25.298000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf13e0f-c482-4a01-a286-1cdb55fa8934 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f2b6dc3406f46e68c74677f5fc7e3b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:25.300619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf13e0f-c482-4a01-a286-1cdb55fa8934 service glance] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:25.301228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf13e0f-c482-4a01-a286-1cdb55fa8934 service glance] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:25.305249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf13e0f-c482-4a01-a286-1cdb55fa8934 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:25.328059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf13e0f-c482-4a01-a286-1cdb55fa8934 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:25.329796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf13e0f-c482-4a01-a286-1cdb55fa8934 service glance] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:25.363108 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1131/2252] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:26.375082 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f256f6-d0ba-43fe-b14d-3934dfa56905 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:26.375871 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f256f6-d0ba-43fe-b14d-3934dfa56905 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:26.376671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f256f6-d0ba-43fe-b14d-3934dfa56905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:26.549039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-c6f256f6-d0ba-43fe-b14d-3934dfa56905 None None] MFA Rules not processed for user `2ca1c6464b6d432ab1bd3898c69a3e47`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:26.552350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c6f256f6-d0ba-43fe-b14d-3934dfa56905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:26.846796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:26.888919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1122/2253] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 516 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:26.949843 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf13e0f-c482-4a01-a286-1cdb55fa8934 service glance] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:26.955143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-daf13e0f-c482-4a01-a286-1cdb55fa8934 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:27.168218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf13e0f-c482-4a01-a286-1cdb55fa8934 service glance] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:27.193788 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e365d8-9765-4ec4-81d9-37921188e385 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:27.194419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e365d8-9765-4ec4-81d9-37921188e385 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:27.194700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e365d8-9765-4ec4-81d9-37921188e385 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:27.194976 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e365d8-9765-4ec4-81d9-37921188e385 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:27.215409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e365d8-9765-4ec4-81d9-37921188e385 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:27.216831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e365d8-9765-4ec4-81d9-37921188e385 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:27.225428 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:27.247657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1132/2254] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:29.011259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0651163-5baf-4806-b2ae-b0f0fa745686 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:29.011851 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0651163-5baf-4806-b2ae-b0f0fa745686 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:29.012088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0651163-5baf-4806-b2ae-b0f0fa745686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:29.043381 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-b0651163-5baf-4806-b2ae-b0f0fa745686 None None] MFA Rules not processed for user `2ca1c6464b6d432ab1bd3898c69a3e47`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:29.045799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b0651163-5baf-4806-b2ae-b0f0fa745686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:29.060183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1123/2255] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:31.403765 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3d31a-bb63-49d8-b1b5-996b91485e3b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:31.404339 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3d31a-bb63-49d8-b1b5-996b91485e3b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:31.404715 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3d31a-bb63-49d8-b1b5-996b91485e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:31.462871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-ead3d31a-bb63-49d8-b1b5-996b91485e3b None None] MFA Rules not processed for user `8ef862d6f6674114b783993006f68258`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:31.464437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ead3d31a-bb63-49d8-b1b5-996b91485e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:31.557403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1133/2256] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:31.584845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0651163-5baf-4806-b2ae-b0f0fa745686 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:53:31.589061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b0651163-5baf-4806-b2ae-b0f0fa745686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:31.711160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0651163-5baf-4806-b2ae-b0f0fa745686 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:53:31.717848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af52c4-ec78-4a3d-a493-d67e06f8a367 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:53:31.718608 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af52c4-ec78-4a3d-a493-d67e06f8a367 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:31.718875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af52c4-ec78-4a3d-a493-d67e06f8a367 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:31.719406 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af52c4-ec78-4a3d-a493-d67e06f8a367 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:31.721292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-42af52c4-ec78-4a3d-a493-d67e06f8a367 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:31.808379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af52c4-ec78-4a3d-a493-d67e06f8a367 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:53:31.810015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af52c4-ec78-4a3d-a493-d67e06f8a367 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:53:31.820251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1124/2257] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:53:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:53:41.554010 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=35 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:41.816106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=61 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:53:47.351416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b695c-927f-413c-ac32-a1045d15e644 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:53:47.352065 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b695c-927f-413c-ac32-a1045d15e644 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:53:47.352065 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b695c-927f-413c-ac32-a1045d15e644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:53:47.372608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3a6b695c-927f-413c-ac32-a1045d15e644 None None] MFA Rules not processed for user `36dbc938bcd741f2a2b2013b6aa6f008`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:53:47.373894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3a6b695c-927f-413c-ac32-a1045d15e644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:53:47.441494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1134/2258] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:53:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:53:57.428872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=25 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:54:03.152542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d81b2a-99bf-41dd-b8f3-24d6ec77f10a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:03.153152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d81b2a-99bf-41dd-b8f3-24d6ec77f10a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:03.153283 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d81b2a-99bf-41dd-b8f3-24d6ec77f10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:03.181512 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-54d81b2a-99bf-41dd-b8f3-24d6ec77f10a None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:54:03.181512 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-54d81b2a-99bf-41dd-b8f3-24d6ec77f10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:03.241659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1125/2259] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:54:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:54:03.299257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b695c-927f-413c-ac32-a1045d15e644 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:03.300786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3a6b695c-927f-413c-ac32-a1045d15e644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:03.411729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b695c-927f-413c-ac32-a1045d15e644 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:03.428179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a9ea-523a-478b-abb7-8e745fe47a78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:03.428366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a9ea-523a-478b-abb7-8e745fe47a78 service swift] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:03.428778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a9ea-523a-478b-abb7-8e745fe47a78 service swift] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:03.429062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a9ea-523a-478b-abb7-8e745fe47a78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:03.439509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0a9ea-523a-478b-abb7-8e745fe47a78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:03.441172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0a9ea-523a-478b-abb7-8e745fe47a78 service swift] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:03.450158 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1135/2260] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:54:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:54:10.416422 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba213a-ddcf-4d2c-be95-f72119819084 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:10.416965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba213a-ddcf-4d2c-be95-f72119819084 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:10.416965 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba213a-ddcf-4d2c-be95-f72119819084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:10.437135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-98ba213a-ddcf-4d2c-be95-f72119819084 None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:54:10.438381 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-98ba213a-ddcf-4d2c-be95-f72119819084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:10.446801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1126/2261] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:54:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:54:10.464054 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a9ea-523a-478b-abb7-8e745fe47a78 service swift] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:10.475838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a9ea-523a-478b-abb7-8e745fe47a78 service swift] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:10.486223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca01b3-a52f-4bc9-ac51-6092441594ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:10.486223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca01b3-a52f-4bc9-ac51-6092441594ee service swift] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:10.486223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca01b3-a52f-4bc9-ac51-6092441594ee service swift] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:10.486223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca01b3-a52f-4bc9-ac51-6092441594ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:10.492932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca01b3-a52f-4bc9-ac51-6092441594ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:10.494522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca01b3-a52f-4bc9-ac51-6092441594ee service swift] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:10.503320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1136/2262] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:54:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:54:11.807525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0850ab-b432-4399-9344-180c5b24986f None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:11.808321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0850ab-b432-4399-9344-180c5b24986f None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:11.808321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0850ab-b432-4399-9344-180c5b24986f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:11.870070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-5e0850ab-b432-4399-9344-180c5b24986f None None] MFA Rules not processed for user `ac4d918f549c41d3ae19e80cecefa95c`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:54:11.871514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5e0850ab-b432-4399-9344-180c5b24986f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:11.901567 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1127/2263] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:54:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:54:17.188114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca01b3-a52f-4bc9-ac51-6092441594ee service swift] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:17.452159 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca01b3-a52f-4bc9-ac51-6092441594ee service swift] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:17.474034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:17.474706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:17.474985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:17.475260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] PATH_INFO: `/v3/users/757c68d99b4c4bf9bb872070f6d83db6` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:17.486305 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=757c68d99b4c4bf9bb872070f6d83db6)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:17.493179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:17.644134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '757c68d99b4c4bf9bb872070f6d83db6'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:17.666388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '757c68d99b4c4bf9bb872070f6d83db6'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:17.676224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '757c68d99b4c4bf9bb872070f6d83db6'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:17.702198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '757c68d99b4c4bf9bb872070f6d83db6'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:17.713256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:54:17.721342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '757c68d99b4c4bf9bb872070f6d83db6'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:17.734231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1137/2264] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:54:17 2022] DELETE /identity/v3/users/757c68d99b4c4bf9bb872070f6d83db6 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:54:17.735807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10d02c-782c-467b-a1ca-9b60a287e11e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:17.736325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10d02c-782c-467b-a1ca-9b60a287e11e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:17.736416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10d02c-782c-467b-a1ca-9b60a287e11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:17.803165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-9c10d02c-782c-467b-a1ca-9b60a287e11e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:54:17.804681 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9c10d02c-782c-467b-a1ca-9b60a287e11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:18.111239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1128/2265] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:54:18.129722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:18.131364 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:18.398588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9b7e5-c8c3-44fc-9f49-6827371a2a1c admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:18.398588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f35578-d8df-4fd7-b647-885a2c3e745c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:18.398588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f35578-d8df-4fd7-b647-885a2c3e745c service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:18.398588 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f35578-d8df-4fd7-b647-885a2c3e745c service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:18.399666 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f35578-d8df-4fd7-b647-885a2c3e745c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:18.444916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f35578-d8df-4fd7-b647-885a2c3e745c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:18.446605 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f35578-d8df-4fd7-b647-885a2c3e745c service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:18.521503 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1138/2266] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:54:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:54:19.156945 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10d02c-782c-467b-a1ca-9b60a287e11e None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:19.159082 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9c10d02c-782c-467b-a1ca-9b60a287e11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:19.192888 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10d02c-782c-467b-a1ca-9b60a287e11e None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:19.209597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:19.210306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:19.211001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:19.211482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] PATH_INFO: `/v3/projects/49606056b6544cbabc16d65cb30b4c70` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:19.213397 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49606056b6544cbabc16d65cb30b4c70)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:19.215357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:19.329991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49606056b6544cbabc16d65cb30b4c70', 'request_id': 'req-ea531f56-54b8-43b0-8167-14eb12126422'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:19.349119 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49606056b6544cbabc16d65cb30b4c70', 'request_id': 'req-ea531f56-54b8-43b0-8167-14eb12126422'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:19.366915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49606056b6544cbabc16d65cb30b4c70', 'request_id': 'req-ea531f56-54b8-43b0-8167-14eb12126422'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:19.387207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] The token cache is being invalidate because project 49606056b6544cbabc16d65cb30b4c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:54:19.390008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:19.393285 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:19.412594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1129/2267] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:54:19 2022] DELETE /identity/v3/projects/49606056b6544cbabc16d65cb30b4c70 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:54:28.463394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=28 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:54:29.380736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:54:32.965101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ab62d-be31-462c-b35d-fe54e2d74b68 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:32.965612 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ab62d-be31-462c-b35d-fe54e2d74b68 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:32.965612 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ab62d-be31-462c-b35d-fe54e2d74b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:32.990950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-631ab62d-be31-462c-b35d-fe54e2d74b68 None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:54:32.992778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-631ab62d-be31-462c-b35d-fe54e2d74b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:33.057377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1139/2268] 10.222.0.36 () {58 vars in 967 bytes} [Thu Aug 18 14:54:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:54:33.071384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:33.073684 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:33.153634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea531f56-54b8-43b0-8167-14eb12126422 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:33.159325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda906b-f4ec-4834-8fe0-12bfc2bc5760 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:33.160075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda906b-f4ec-4834-8fe0-12bfc2bc5760 service swift] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:33.160426 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda906b-f4ec-4834-8fe0-12bfc2bc5760 service swift] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:33.160774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda906b-f4ec-4834-8fe0-12bfc2bc5760 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:33.170765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda906b-f4ec-4834-8fe0-12bfc2bc5760 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:33.176011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda906b-f4ec-4834-8fe0-12bfc2bc5760 service swift] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:33.185008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1130/2269] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:54:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:54:35.293607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ab62d-be31-462c-b35d-fe54e2d74b68 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:35.295046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-631ab62d-be31-462c-b35d-fe54e2d74b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:35.444820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ab62d-be31-462c-b35d-fe54e2d74b68 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:35.457975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7621f4-7fe0-4d17-8857-a26d90ce092c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:35.458652 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7621f4-7fe0-4d17-8857-a26d90ce092c service placement] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:35.458940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7621f4-7fe0-4d17-8857-a26d90ce092c service placement] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:35.459213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7621f4-7fe0-4d17-8857-a26d90ce092c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:35.465509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8c7621f4-7fe0-4d17-8857-a26d90ce092c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:35.501945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7621f4-7fe0-4d17-8857-a26d90ce092c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:35.503578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7621f4-7fe0-4d17-8857-a26d90ce092c service placement] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:35.527226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1140/2270] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:54:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:54:35.566142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda906b-f4ec-4834-8fe0-12bfc2bc5760 service swift] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:35.567707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bfda906b-f4ec-4834-8fe0-12bfc2bc5760 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:35.727818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda906b-f4ec-4834-8fe0-12bfc2bc5760 service swift] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:35.732820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110873fb-a0df-47f9-9933-539e2d7c5032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:35.733442 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110873fb-a0df-47f9-9933-539e2d7c5032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:35.733719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110873fb-a0df-47f9-9933-539e2d7c5032 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:35.733985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110873fb-a0df-47f9-9933-539e2d7c5032 admin admin] PATH_INFO: `/v3/users/ae0dfb7613e1432dacbfc6011efaa0ae` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:35.735094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110873fb-a0df-47f9-9933-539e2d7c5032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0dfb7613e1432dacbfc6011efaa0ae)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:35.737153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110873fb-a0df-47f9-9933-539e2d7c5032 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:35.793216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-110873fb-a0df-47f9-9933-539e2d7c5032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0dfb7613e1432dacbfc6011efaa0ae'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:35.803031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-110873fb-a0df-47f9-9933-539e2d7c5032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0dfb7613e1432dacbfc6011efaa0ae'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:35.804499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-110873fb-a0df-47f9-9933-539e2d7c5032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0dfb7613e1432dacbfc6011efaa0ae'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:35.805200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-110873fb-a0df-47f9-9933-539e2d7c5032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0dfb7613e1432dacbfc6011efaa0ae'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:35.822616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-110873fb-a0df-47f9-9933-539e2d7c5032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0dfb7613e1432dacbfc6011efaa0ae'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:35.839809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1131/2271] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:54:35 2022] DELETE /identity/v3/users/ae0dfb7613e1432dacbfc6011efaa0ae => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:54:36.070828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7621f4-7fe0-4d17-8857-a26d90ce092c service placement] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:36.087729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7621f4-7fe0-4d17-8857-a26d90ce092c service placement] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:36.166521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:36.167329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:36.167696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:36.168116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e admin admin] PATH_INFO: `/v3/projects/caf9864085c74207bd586b9493d91fe2` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:36.169879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e admin admin] RBAC: Authorizing `identity:delete_project(project_id=caf9864085c74207bd586b9493d91fe2)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:36.172007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:36.220507 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:54:36.235780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caf9864085c74207bd586b9493d91fe2', 'request_id': 'req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:36.247761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caf9864085c74207bd586b9493d91fe2', 'request_id': 'req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:36.270610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caf9864085c74207bd586b9493d91fe2', 'request_id': 'req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:36.294276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e admin admin] The token cache is being invalidate because project caf9864085c74207bd586b9493d91fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:54:36.294877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:36.296489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-24c2f85b-0f7c-48ae-84d0-7c5770b1c84e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:54:36.300645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1141/2272] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:54:36 2022] DELETE /identity/v3/projects/caf9864085c74207bd586b9493d91fe2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:54:36.307699 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28082d27-0383-4522-bce9-79476ba446e0 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:36.308048 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28082d27-0383-4522-bce9-79476ba446e0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:36.308487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28082d27-0383-4522-bce9-79476ba446e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:36.337506 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-28082d27-0383-4522-bce9-79476ba446e0 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:54:36.339383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-28082d27-0383-4522-bce9-79476ba446e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:36.343958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:54:36.414746 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1132/2273] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:54:36.422885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8bc91-412a-4dc2-8fe2-97a297f5ced2 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:36.423176 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8bc91-412a-4dc2-8fe2-97a297f5ced2 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:36.423453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8bc91-412a-4dc2-8fe2-97a297f5ced2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:36.445124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-c5e8bc91-412a-4dc2-8fe2-97a297f5ced2 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:54:36.446480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c5e8bc91-412a-4dc2-8fe2-97a297f5ced2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:36.455176 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1142/2274] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:54:36.460060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28082d27-0383-4522-bce9-79476ba446e0 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:36.473416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28082d27-0383-4522-bce9-79476ba446e0 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:36.480165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cc7af-66ee-4067-94b3-72ab9147b307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:36.480781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cc7af-66ee-4067-94b3-72ab9147b307 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:36.481058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cc7af-66ee-4067-94b3-72ab9147b307 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:36.481333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cc7af-66ee-4067-94b3-72ab9147b307 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:36.481982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cc7af-66ee-4067-94b3-72ab9147b307 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:54:36.482285 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cc7af-66ee-4067-94b3-72ab9147b307 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:36.483949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cc7af-66ee-4067-94b3-72ab9147b307 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:36.501110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1133/2275] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:54:36.502244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8bc91-412a-4dc2-8fe2-97a297f5ced2 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:36.512606 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8bc91-412a-4dc2-8fe2-97a297f5ced2 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:36.517795 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40553114-7b97-4206-9226-e65089981a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:36.518516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40553114-7b97-4206-9226-e65089981a01 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:36.518816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40553114-7b97-4206-9226-e65089981a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:36.519093 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40553114-7b97-4206-9226-e65089981a01 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:36.519831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40553114-7b97-4206-9226-e65089981a01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:36.521783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40553114-7b97-4206-9226-e65089981a01 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:36.555828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1143/2276] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:54:36 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:54:36.559942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cc7af-66ee-4067-94b3-72ab9147b307 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:36.571421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cc7af-66ee-4067-94b3-72ab9147b307 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:36.656020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962c35f-e7a2-4d12-941e-a1be555ef8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:36.658299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962c35f-e7a2-4d12-941e-a1be555ef8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:36.658793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962c35f-e7a2-4d12-941e-a1be555ef8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:36.659225 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962c35f-e7a2-4d12-941e-a1be555ef8b0 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:36.660326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962c35f-e7a2-4d12-941e-a1be555ef8b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:36.663137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962c35f-e7a2-4d12-941e-a1be555ef8b0 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:36.700731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1134/2277] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:54:36 2022] POST /identity/v3/users => generated 296 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:54:36.704765 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40553114-7b97-4206-9226-e65089981a01 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:36.717951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40553114-7b97-4206-9226-e65089981a01 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:36.723154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4262708-cf83-421c-bf92-d96ea88f7866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:36.723862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4262708-cf83-421c-bf92-d96ea88f7866 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:36.724397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4262708-cf83-421c-bf92-d96ea88f7866 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:36.725154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4262708-cf83-421c-bf92-d96ea88f7866 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:36.726033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4262708-cf83-421c-bf92-d96ea88f7866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:36.728000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4262708-cf83-421c-bf92-d96ea88f7866 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:36.740200 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1144/2278] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:54:36.745209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962c35f-e7a2-4d12-941e-a1be555ef8b0 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:36.755644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962c35f-e7a2-4d12-941e-a1be555ef8b0 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:36.761156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10290238-fa4c-4f78-b453-9d0e9d57c639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:36.761761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10290238-fa4c-4f78-b453-9d0e9d57c639 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:36.762038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10290238-fa4c-4f78-b453-9d0e9d57c639 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:36.762308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10290238-fa4c-4f78-b453-9d0e9d57c639 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:36.762946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10290238-fa4c-4f78-b453-9d0e9d57c639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:36.764584 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10290238-fa4c-4f78-b453-9d0e9d57c639 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:36.776072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1135/2279] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:54:36.780396 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4262708-cf83-421c-bf92-d96ea88f7866 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:36.791609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4262708-cf83-421c-bf92-d96ea88f7866 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:36.797075 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63399ac0-73b2-4fad-8726-2ddddea5bad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:36.798031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63399ac0-73b2-4fad-8726-2ddddea5bad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:36.798537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63399ac0-73b2-4fad-8726-2ddddea5bad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:36.798951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63399ac0-73b2-4fad-8726-2ddddea5bad0 admin admin] PATH_INFO: `/v3/projects/304ab0126c8a4d8ca127c8e878f7d1d8/users/0d1939a74bd74feda39d47fb24acc6dc/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:36.816932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63399ac0-73b2-4fad-8726-2ddddea5bad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=304ab0126c8a4d8ca127c8e878f7d1d8, user_id=0d1939a74bd74feda39d47fb24acc6dc, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:36.820038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63399ac0-73b2-4fad-8726-2ddddea5bad0 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:36.835787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1145/2280] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:54:36 2022] PUT /identity/v3/projects/304ab0126c8a4d8ca127c8e878f7d1d8/users/0d1939a74bd74feda39d47fb24acc6dc/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:54:36.847697 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e99da2-25b4-4892-b98e-4cb002bd7041 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:36.848059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e99da2-25b4-4892-b98e-4cb002bd7041 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:36.848676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e99da2-25b4-4892-b98e-4cb002bd7041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:36.903263 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-d6e99da2-25b4-4892-b98e-4cb002bd7041 None None] MFA Rules not processed for user `0d1939a74bd74feda39d47fb24acc6dc`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:54:36.904484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d6e99da2-25b4-4892-b98e-4cb002bd7041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:36.910676 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e9aa6-4ae9-45dc-9bac-502d0a2af835 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:36.910966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e9aa6-4ae9-45dc-9bac-502d0a2af835 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:36.911227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e9aa6-4ae9-45dc-9bac-502d0a2af835 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:36.912645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1146/2281] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 14:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:54:36.920520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b21826-8ff0-4664-9204-4f27ac14320e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:36.921022 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b21826-8ff0-4664-9204-4f27ac14320e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:36.921362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b21826-8ff0-4664-9204-4f27ac14320e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:36.944635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-15b21826-8ff0-4664-9204-4f27ac14320e None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:54:36.946025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-15b21826-8ff0-4664-9204-4f27ac14320e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:36.988220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1136/2282] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:54:37.010127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e99da2-25b4-4892-b98e-4cb002bd7041 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:37.012239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d6e99da2-25b4-4892-b98e-4cb002bd7041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:37.020738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:54:37.034310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1147/2283] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:54:37.049597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b21826-8ff0-4664-9204-4f27ac14320e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:37.050046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-15b21826-8ff0-4664-9204-4f27ac14320e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:37.093276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e99da2-25b4-4892-b98e-4cb002bd7041 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:37.097492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb571b0-1b2a-416f-8e03-ce2c9eebd42a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:37.098112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb571b0-1b2a-416f-8e03-ce2c9eebd42a service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:37.098719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb571b0-1b2a-416f-8e03-ce2c9eebd42a service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:37.098832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb571b0-1b2a-416f-8e03-ce2c9eebd42a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:37.108924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb571b0-1b2a-416f-8e03-ce2c9eebd42a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:37.110618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb571b0-1b2a-416f-8e03-ce2c9eebd42a service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:37.121765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1137/2284] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:54:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:54:37.129245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b21826-8ff0-4664-9204-4f27ac14320e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:37.135229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f1cc6-595f-4c7c-9118-d83019b4783f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f2b6dc3406f46e68c74677f5fc7e3b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:37.135886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f1cc6-595f-4c7c-9118-d83019b4783f service glance] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:37.136167 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f1cc6-595f-4c7c-9118-d83019b4783f service glance] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:37.136441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f1cc6-595f-4c7c-9118-d83019b4783f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:37.150657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:54:37.153826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f1cc6-595f-4c7c-9118-d83019b4783f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:37.155899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f1cc6-595f-4c7c-9118-d83019b4783f service glance] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:37.176394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1148/2285] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:54:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:54:37.247480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e95cc-7161-49ee-bc95-b1c3224e47ca None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:37.247875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e95cc-7161-49ee-bc95-b1c3224e47ca None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:37.248154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e95cc-7161-49ee-bc95-b1c3224e47ca None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:37.249485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1138/2286] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 14:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:54:37.254471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfc3e2-8f4a-4ead-a7ff-c2726ca54c11 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:37.254752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfc3e2-8f4a-4ead-a7ff-c2726ca54c11 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:37.255025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfc3e2-8f4a-4ead-a7ff-c2726ca54c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:37.275779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-1fdfc3e2-8f4a-4ead-a7ff-c2726ca54c11 None None] MFA Rules not processed for user `00f209d8426a49acaca7e9d00a1b9ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:54:37.276944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-1fdfc3e2-8f4a-4ead-a7ff-c2726ca54c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:37.337061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1149/2287] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:54:37.349787 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e95cc-7161-49ee-bc95-b1c3224e47ca None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:37.352155 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-fe9e95cc-7161-49ee-bc95-b1c3224e47ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:37.365924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e95cc-7161-49ee-bc95-b1c3224e47ca None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:37.369820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75e75a-c7ca-40a8-9fa1-7821439b0e9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:37.370432 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75e75a-c7ca-40a8-9fa1-7821439b0e9d service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:37.371129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75e75a-c7ca-40a8-9fa1-7821439b0e9d service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:37.371364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75e75a-c7ca-40a8-9fa1-7821439b0e9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:37.378976 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75e75a-c7ca-40a8-9fa1-7821439b0e9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:37.380468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75e75a-c7ca-40a8-9fa1-7821439b0e9d service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:37.390127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1139/2288] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:54:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:54:37.492070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a8f02-7489-484d-9a50-02b3f34b8555 None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:37.492362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a8f02-7489-484d-9a50-02b3f34b8555 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:37.492645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a8f02-7489-484d-9a50-02b3f34b8555 None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:37.494001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1150/2289] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 14:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:54:37.499642 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d253eb-d691-48d3-9594-eeb04a830882 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:37.500029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d253eb-d691-48d3-9594-eeb04a830882 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:37.500395 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d253eb-d691-48d3-9594-eeb04a830882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:37.521299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-c4d253eb-d691-48d3-9594-eeb04a830882 None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:54:37.522515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c4d253eb-d691-48d3-9594-eeb04a830882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:37.532646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1140/2290] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:54:37.543562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a8f02-7489-484d-9a50-02b3f34b8555 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:37.554526 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a8f02-7489-484d-9a50-02b3f34b8555 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:37.558061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d83fa-e65c-4dc6-8c02-36d5a7f60ccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:37.558648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d83fa-e65c-4dc6-8c02-36d5a7f60ccb service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:37.558926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d83fa-e65c-4dc6-8c02-36d5a7f60ccb service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:37.559201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d83fa-e65c-4dc6-8c02-36d5a7f60ccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:37.569156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d83fa-e65c-4dc6-8c02-36d5a7f60ccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:37.571148 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d83fa-e65c-4dc6-8c02-36d5a7f60ccb service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:37.583276 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1151/2291] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:54:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:54:39.569759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d253eb-d691-48d3-9594-eeb04a830882 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:39.578896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d253eb-d691-48d3-9594-eeb04a830882 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:39.582626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b60ae-ea6d-48d4-a067-ef4c5c747ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:39.583221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b60ae-ea6d-48d4-a067-ef4c5c747ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:39.583526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b60ae-ea6d-48d4-a067-ef4c5c747ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:39.583821 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b60ae-ea6d-48d4-a067-ef4c5c747ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:39.590248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e43b60ae-ea6d-48d4-a067-ef4c5c747ce6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:39.654443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b60ae-ea6d-48d4-a067-ef4c5c747ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:39.655911 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b60ae-ea6d-48d4-a067-ef4c5c747ce6 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:39.666341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1141/2292] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:54:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:54:47.578841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=31 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:54:49.661238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=63 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:54:54.048169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126e56c-e602-4d62-9fd8-33c25221ac7b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:54.048710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126e56c-e602-4d62-9fd8-33c25221ac7b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:54.048710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126e56c-e602-4d62-9fd8-33c25221ac7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:54.069267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-1126e56c-e602-4d62-9fd8-33c25221ac7b None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:54:54.071198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-1126e56c-e602-4d62-9fd8-33c25221ac7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:54.133333 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1152/2293] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:54:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:54:54.148536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b60ae-ea6d-48d4-a067-ef4c5c747ce6 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:54:54.150109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e43b60ae-ea6d-48d4-a067-ef4c5c747ce6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:54:54.222659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b60ae-ea6d-48d4-a067-ef4c5c747ce6 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:54:54.228958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e18b8-f9b0-43c7-8b96-e36069e2cb93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:54:54.229616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e18b8-f9b0-43c7-8b96-e36069e2cb93 service swift] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:54:54.229890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e18b8-f9b0-43c7-8b96-e36069e2cb93 service swift] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:54:54.230156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e18b8-f9b0-43c7-8b96-e36069e2cb93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:54:54.238932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e18b8-f9b0-43c7-8b96-e36069e2cb93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:54:54.240850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e18b8-f9b0-43c7-8b96-e36069e2cb93 service swift] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:54:54.251589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1142/2294] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:54:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:55:03.971465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126e56c-e602-4d62-9fd8-33c25221ac7b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:55:04.042006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126e56c-e602-4d62-9fd8-33c25221ac7b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:55:04.048868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:55:04.049610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:55:04.049908 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:55:04.050188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] PATH_INFO: `/v3/users/0d1939a74bd74feda39d47fb24acc6dc` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:55:04.051442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d1939a74bd74feda39d47fb24acc6dc)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:55:04.053677 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:55:04.130895 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d1939a74bd74feda39d47fb24acc6dc'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:55:04.138910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d1939a74bd74feda39d47fb24acc6dc'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:55:04.139771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d1939a74bd74feda39d47fb24acc6dc'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:55:04.158012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d1939a74bd74feda39d47fb24acc6dc'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:55:04.167608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d1939a74bd74feda39d47fb24acc6dc'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:55:04.174606 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1153/2295] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:55:03 2022] DELETE /identity/v3/users/0d1939a74bd74feda39d47fb24acc6dc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:55:04.179895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d8afd-9b1d-4567-a428-c534268dd8a8 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:55:04.180955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d8afd-9b1d-4567-a428-c534268dd8a8 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:55:04.181231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d8afd-9b1d-4567-a428-c534268dd8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:55:04.203593 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-605d8afd-9b1d-4567-a428-c534268dd8a8 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:55:04.205367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-605d8afd-9b1d-4567-a428-c534268dd8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:55:04.267780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1143/2296] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:55:04.278786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:55:04.280266 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:55:04.308646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:55:04.342097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b979d-518e-4650-a536-ffde2a675450 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:55:04.345848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7c654-fb3a-4378-9292-bf47629afe09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:55:04.346456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7c654-fb3a-4378-9292-bf47629afe09 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:55:04.346742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7c654-fb3a-4378-9292-bf47629afe09 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:55:04.347014 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7c654-fb3a-4378-9292-bf47629afe09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:55:04.358975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7c654-fb3a-4378-9292-bf47629afe09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:55:04.360564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7c654-fb3a-4378-9292-bf47629afe09 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:55:04.371099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1154/2297] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:55:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:55:04.638367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d8afd-9b1d-4567-a428-c534268dd8a8 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:55:04.639851 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-605d8afd-9b1d-4567-a428-c534268dd8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:55:04.654072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d8afd-9b1d-4567-a428-c534268dd8a8 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:55:04.659431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d5481-b5a6-4497-98e4-f329ed751170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:55:04.660095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d5481-b5a6-4497-98e4-f329ed751170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:55:04.660371 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d5481-b5a6-4497-98e4-f329ed751170 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:55:04.660637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d5481-b5a6-4497-98e4-f329ed751170 admin admin] PATH_INFO: `/v3/projects/304ab0126c8a4d8ca127c8e878f7d1d8` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:55:04.661805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d5481-b5a6-4497-98e4-f329ed751170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=304ab0126c8a4d8ca127c8e878f7d1d8)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:55:04.663878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d5481-b5a6-4497-98e4-f329ed751170 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:55:04.710014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-926d5481-b5a6-4497-98e4-f329ed751170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '304ab0126c8a4d8ca127c8e878f7d1d8', 'request_id': 'req-926d5481-b5a6-4497-98e4-f329ed751170'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:55:04.718262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-926d5481-b5a6-4497-98e4-f329ed751170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '304ab0126c8a4d8ca127c8e878f7d1d8', 'request_id': 'req-926d5481-b5a6-4497-98e4-f329ed751170'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:55:04.725206 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-926d5481-b5a6-4497-98e4-f329ed751170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '304ab0126c8a4d8ca127c8e878f7d1d8', 'request_id': 'req-926d5481-b5a6-4497-98e4-f329ed751170'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:55:04.736889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-926d5481-b5a6-4497-98e4-f329ed751170 admin admin] The token cache is being invalidate because project 304ab0126c8a4d8ca127c8e878f7d1d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:55:04.737176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-926d5481-b5a6-4497-98e4-f329ed751170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:55:04.737805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-926d5481-b5a6-4497-98e4-f329ed751170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:55:04.739294 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1144/2298] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:55:04 2022] DELETE /identity/v3/projects/304ab0126c8a4d8ca127c8e878f7d1d8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:55:14.366550 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=15 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:55:14.733862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:55:17.709699 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a1c71-05bf-4acf-8d6f-f72b1d3ea72a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:55:17.710248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a1c71-05bf-4acf-8d6f-f72b1d3ea72a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:55:17.710344 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a1c71-05bf-4acf-8d6f-f72b1d3ea72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:55:17.728923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-cb6a1c71-05bf-4acf-8d6f-f72b1d3ea72a None None] MFA Rules not processed for user `26eaa7f102bd41028983458ae48ff992`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:55:17.730133 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-cb6a1c71-05bf-4acf-8d6f-f72b1d3ea72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:55:17.784586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1155/2299] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:55:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:55:27.778047 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=25 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:55:28.301510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65135c-2760-4bcf-85fa-1c149f8bb4ba None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:55:28.302059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65135c-2760-4bcf-85fa-1c149f8bb4ba None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:55:28.302239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65135c-2760-4bcf-85fa-1c149f8bb4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:55:28.322842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-5a65135c-2760-4bcf-85fa-1c149f8bb4ba None None] MFA Rules not processed for user `2ca1c6464b6d432ab1bd3898c69a3e47`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:55:28.324064 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5a65135c-2760-4bcf-85fa-1c149f8bb4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:55:28.376950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1145/2300] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:55:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:55:29.433036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b71a77-a46b-4802-835c-65ba7c26e8a7 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:55:29.433494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b71a77-a46b-4802-835c-65ba7c26e8a7 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:55:29.433602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b71a77-a46b-4802-835c-65ba7c26e8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:55:29.460141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-e1b71a77-a46b-4802-835c-65ba7c26e8a7 None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:55:29.460141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e1b71a77-a46b-4802-835c-65ba7c26e8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:55:29.525695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1156/2301] 10.222.0.36 () {58 vars in 967 bytes} [Thu Aug 18 14:55:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:55:29.550721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65135c-2760-4bcf-85fa-1c149f8bb4ba None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:55:29.552253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5a65135c-2760-4bcf-85fa-1c149f8bb4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:55:29.641262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65135c-2760-4bcf-85fa-1c149f8bb4ba None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:55:29.647500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b8942-5082-4b13-b802-e326ac5f0b57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:55:29.649417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b8942-5082-4b13-b802-e326ac5f0b57 service swift] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:55:29.650094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b8942-5082-4b13-b802-e326ac5f0b57 service swift] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:55:29.650094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b8942-5082-4b13-b802-e326ac5f0b57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:55:29.658538 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b8942-5082-4b13-b802-e326ac5f0b57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:55:29.660139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b8942-5082-4b13-b802-e326ac5f0b57 service swift] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:55:29.671362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1146/2302] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:55:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:55:39.518178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=25 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:55:39.667433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=57 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:55:48.116805 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea7449-cb20-487d-a92f-8b2d44a6562b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:55:48.117298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea7449-cb20-487d-a92f-8b2d44a6562b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:55:48.117407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea7449-cb20-487d-a92f-8b2d44a6562b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:55:48.141426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-29ea7449-cb20-487d-a92f-8b2d44a6562b None None] MFA Rules not processed for user `36dbc938bcd741f2a2b2013b6aa6f008`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:55:48.142670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-29ea7449-cb20-487d-a92f-8b2d44a6562b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:55:48.196845 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1157/2303] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:55:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:55:58.190114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=25 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:56:20.448246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b8942-5082-4b13-b802-e326ac5f0b57 service swift] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:56:20.449675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-284b8942-5082-4b13-b802-e326ac5f0b57 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:20.507378 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b8942-5082-4b13-b802-e326ac5f0b57 service swift] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:56:20.511295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a3e5c-f66b-4597-9f8e-667665188cec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:56:20.511933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3e5c-f66b-4597-9f8e-667665188cec service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:20.512213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3e5c-f66b-4597-9f8e-667665188cec service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:20.512488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3e5c-f66b-4597-9f8e-667665188cec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:20.514222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f78a3e5c-f66b-4597-9f8e-667665188cec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:20.598273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a3e5c-f66b-4597-9f8e-667665188cec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:56:20.600312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a3e5c-f66b-4597-9f8e-667665188cec service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:56:20.611174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1147/2304] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:56:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:56:30.606330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=54 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:56:32.177099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5539fe-d0b8-4ed5-b01c-cf8fbcab72ac None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:32.178105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5539fe-d0b8-4ed5-b01c-cf8fbcab72ac None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:32.178211 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5539fe-d0b8-4ed5-b01c-cf8fbcab72ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:32.197370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-ea5539fe-d0b8-4ed5-b01c-cf8fbcab72ac None None] MFA Rules not processed for user `09b055bb15ce4b2ab4276b0090e16122`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:56:32.198589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ea5539fe-d0b8-4ed5-b01c-cf8fbcab72ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:32.252719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1158/2305] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:56:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:56:32.558688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a3e5c-f66b-4597-9f8e-667665188cec service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:56:32.560210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f78a3e5c-f66b-4597-9f8e-667665188cec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:32.617350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a3e5c-f66b-4597-9f8e-667665188cec service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:56:32.621290 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc1b1f-7a19-4a6c-91a7-1fd6c012653a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:56:32.621821 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc1b1f-7a19-4a6c-91a7-1fd6c012653a service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:32.622091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc1b1f-7a19-4a6c-91a7-1fd6c012653a service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:32.622353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc1b1f-7a19-4a6c-91a7-1fd6c012653a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:32.624076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-13bc1b1f-7a19-4a6c-91a7-1fd6c012653a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:32.633163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc1b1f-7a19-4a6c-91a7-1fd6c012653a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:56:32.634564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc1b1f-7a19-4a6c-91a7-1fd6c012653a service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:56:32.642516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1148/2306] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:56:32.979273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5539fe-d0b8-4ed5-b01c-cf8fbcab72ac None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:56:32.980805 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ea5539fe-d0b8-4ed5-b01c-cf8fbcab72ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:33.060642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5539fe-d0b8-4ed5-b01c-cf8fbcab72ac None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:56:33.065659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a4bb4-5950-43d8-a79f-19ff99dc75e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:56:33.066265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a4bb4-5950-43d8-a79f-19ff99dc75e3 service placement] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:33.066545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a4bb4-5950-43d8-a79f-19ff99dc75e3 service placement] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:33.066820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a4bb4-5950-43d8-a79f-19ff99dc75e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:33.069095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8f1a4bb4-5950-43d8-a79f-19ff99dc75e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:33.080751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a4bb4-5950-43d8-a79f-19ff99dc75e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:56:33.083010 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a4bb4-5950-43d8-a79f-19ff99dc75e3 service placement] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:56:33.093479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1159/2307] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:56:37.111130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24204e3-f424-4831-a8c8-fb0d38d1293a None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:37.111738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24204e3-f424-4831-a8c8-fb0d38d1293a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:37.112208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24204e3-f424-4831-a8c8-fb0d38d1293a None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:37.113946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1149/2308] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 14:56:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 14:56:37.140727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c348964-2361-44a6-87ee-bbd3c9d50ff1 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:37.141613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c348964-2361-44a6-87ee-bbd3c9d50ff1 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:37.141888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c348964-2361-44a6-87ee-bbd3c9d50ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:37.163471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-9c348964-2361-44a6-87ee-bbd3c9d50ff1 None None] MFA Rules not processed for user `db581b10d78a4f9eae33fadce0d92b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:56:37.164755 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9c348964-2361-44a6-87ee-bbd3c9d50ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:37.177127 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24204e3-f424-4831-a8c8-fb0d38d1293a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:56:37.178914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b24204e3-f424-4831-a8c8-fb0d38d1293a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:37.179377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1160/2309] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 14:56:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:56:37.198826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c348964-2361-44a6-87ee-bbd3c9d50ff1 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:56:37.210313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c348964-2361-44a6-87ee-bbd3c9d50ff1 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:56:37.217232 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587b0a1-0941-42e4-a07c-c8d4b8efb905 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:56:37.217830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587b0a1-0941-42e4-a07c-c8d4b8efb905 service placement] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:37.218109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587b0a1-0941-42e4-a07c-c8d4b8efb905 service placement] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:37.218384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587b0a1-0941-42e4-a07c-c8d4b8efb905 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:37.228125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587b0a1-0941-42e4-a07c-c8d4b8efb905 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:56:37.230311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587b0a1-0941-42e4-a07c-c8d4b8efb905 service placement] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:56:37.241368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1161/2310] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:56:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:56:37.257751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24204e3-f424-4831-a8c8-fb0d38d1293a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:56:37.262732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:56:37.263436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:37.263739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:37.264011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] PATH_INFO: `/v3/users/2ca1c6464b6d432ab1bd3898c69a3e47` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:37.265175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca1c6464b6d432ab1bd3898c69a3e47)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:56:37.267377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:56:37.322867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca1c6464b6d432ab1bd3898c69a3e47'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:56:37.335505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca1c6464b6d432ab1bd3898c69a3e47'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:56:37.336648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca1c6464b6d432ab1bd3898c69a3e47'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:56:37.337680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca1c6464b6d432ab1bd3898c69a3e47'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:56:37.353458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca1c6464b6d432ab1bd3898c69a3e47'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:56:37.366585 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1150/2311] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:56:37 2022] DELETE /identity/v3/users/2ca1c6464b6d432ab1bd3898c69a3e47 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:56:37.545607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587b0a1-0941-42e4-a07c-c8d4b8efb905 service placement] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:56:37.556775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587b0a1-0941-42e4-a07c-c8d4b8efb905 service placement] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:56:37.619378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:56:37.620046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:37.620317 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:37.620584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] PATH_INFO: `/v3/projects/49b5f78066ec41e98faa23bacd4881cf` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:37.621683 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b5f78066ec41e98faa23bacd4881cf)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:56:37.623561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:56:37.641182 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:56:37.679596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b5f78066ec41e98faa23bacd4881cf', 'request_id': 'req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:56:37.685814 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b5f78066ec41e98faa23bacd4881cf', 'request_id': 'req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:56:37.695311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b5f78066ec41e98faa23bacd4881cf', 'request_id': 'req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:56:37.707312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] The token cache is being invalidate because project 49b5f78066ec41e98faa23bacd4881cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:56:37.707621 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:56:37.708331 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:56:37.717064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1162/2312] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:56:37 2022] DELETE /identity/v3/projects/49b5f78066ec41e98faa23bacd4881cf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:56:40.832046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:56:40.834120 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:40.862503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:56:40.916517 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504714e-f013-402f-b029-3ce2cf874394 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:56:40.982605 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cda7ee-f0bf-4fcf-958a-e9ef73c98e8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:56:40.983216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cda7ee-f0bf-4fcf-958a-e9ef73c98e8d service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:40.983495 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cda7ee-f0bf-4fcf-958a-e9ef73c98e8d service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:40.983797 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cda7ee-f0bf-4fcf-958a-e9ef73c98e8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:40.985838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f3cda7ee-f0bf-4fcf-958a-e9ef73c98e8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:41.042647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cda7ee-f0bf-4fcf-958a-e9ef73c98e8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:56:41.044286 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cda7ee-f0bf-4fcf-958a-e9ef73c98e8d service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:56:41.053092 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1151/2313] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:56:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:56:41.317170 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:56:41.318635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:41.334199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9e716-1b8d-4d3d-83d1-3e1d429745a5 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:56:41.339540 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6689a8-70aa-412d-99ee-4cf23be6323d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:56:41.340167 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6689a8-70aa-412d-99ee-4cf23be6323d service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:41.340435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6689a8-70aa-412d-99ee-4cf23be6323d service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:41.340699 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6689a8-70aa-412d-99ee-4cf23be6323d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:41.342415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4f6689a8-70aa-412d-99ee-4cf23be6323d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:41.405672 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6689a8-70aa-412d-99ee-4cf23be6323d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:56:41.407274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6689a8-70aa-412d-99ee-4cf23be6323d service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:56:41.417256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1163/2314] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:56:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:56:42.105586 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cda7ee-f0bf-4fcf-958a-e9ef73c98e8d service nova] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:56:42.107215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f3cda7ee-f0bf-4fcf-958a-e9ef73c98e8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:42.189342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cda7ee-f0bf-4fcf-958a-e9ef73c98e8d service nova] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:56:42.195156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f4398-e90b-48c4-a28d-2cb427042431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:56:42.195824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f4398-e90b-48c4-a28d-2cb427042431 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:42.196111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f4398-e90b-48c4-a28d-2cb427042431 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:42.196382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f4398-e90b-48c4-a28d-2cb427042431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:42.198302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2e1f4398-e90b-48c4-a28d-2cb427042431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:42.210320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f4398-e90b-48c4-a28d-2cb427042431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:56:42.212065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f4398-e90b-48c4-a28d-2cb427042431 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:56:42.216907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:56:42.235736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1152/2315] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:56:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:56:45.446745 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f286b-0d2b-4ac0-ae99-94e3b4d2908e None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:45.447292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f286b-0d2b-4ac0-ae99-94e3b4d2908e None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:45.447292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f286b-0d2b-4ac0-ae99-94e3b4d2908e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:45.467975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-a87f286b-0d2b-4ac0-ae99-94e3b4d2908e None None] MFA Rules not processed for user `4d96208474754b2381d769d807689bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:56:45.469234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a87f286b-0d2b-4ac0-ae99-94e3b4d2908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:45.477143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1164/2316] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:56:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:56:53.071166 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f4398-e90b-48c4-a28d-2cb427042431 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:56:53.085425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f4398-e90b-48c4-a28d-2cb427042431 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:56:53.090803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4056c58-e4bc-4d43-91f5-2da651603ea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:56:53.091390 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4056c58-e4bc-4d43-91f5-2da651603ea3 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:56:53.091664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4056c58-e4bc-4d43-91f5-2da651603ea3 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:56:53.091959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4056c58-e4bc-4d43-91f5-2da651603ea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:56:53.093676 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a4056c58-e4bc-4d43-91f5-2da651603ea3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:56:53.152160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4056c58-e4bc-4d43-91f5-2da651603ea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:56:53.153852 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4056c58-e4bc-4d43-91f5-2da651603ea3 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:56:53.163365 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1153/2317] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:56:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:56:55.462117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:57:00.085399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f286b-0d2b-4ac0-ae99-94e3b4d2908e None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:57:00.099095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f286b-0d2b-4ac0-ae99-94e3b4d2908e None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:57:00.104716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c1e45-c9e2-4f09-973f-b7764e1f29b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:57:00.105320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c1e45-c9e2-4f09-973f-b7764e1f29b3 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:57:00.105665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c1e45-c9e2-4f09-973f-b7764e1f29b3 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:57:00.105878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c1e45-c9e2-4f09-973f-b7764e1f29b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:57:00.107648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7b2c1e45-c9e2-4f09-973f-b7764e1f29b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:57:00.120135 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c1e45-c9e2-4f09-973f-b7764e1f29b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:57:00.121736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c1e45-c9e2-4f09-973f-b7764e1f29b3 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:57:00.132788 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1165/2318] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:57:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:57:03.158868 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=28 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:57:03.422705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c98348-52d0-4e84-8880-c0612c308ecd None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:57:03.422915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c98348-52d0-4e84-8880-c0612c308ecd None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:57:03.423217 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c98348-52d0-4e84-8880-c0612c308ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:57:03.443161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-37c98348-52d0-4e84-8880-c0612c308ecd None None] MFA Rules not processed for user `26eaa7f102bd41028983458ae48ff992`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:57:03.444304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-37c98348-52d0-4e84-8880-c0612c308ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:57:03.502193 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1154/2319] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:57:10.127860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=6 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:57:13.403808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d6fb2-7edb-4708-ba31-b074c27bb19a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:57:13.404460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d6fb2-7edb-4708-ba31-b074c27bb19a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:57:13.404460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d6fb2-7edb-4708-ba31-b074c27bb19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:57:13.426063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-2e9d6fb2-7edb-4708-ba31-b074c27bb19a None None] MFA Rules not processed for user `ac4d918f549c41d3ae19e80cecefa95c`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:57:13.428501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2e9d6fb2-7edb-4708-ba31-b074c27bb19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:57:13.496075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=25 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:57:13.496640 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1166/2320] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:57:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:57:14.921376 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd987c8a-65b8-403b-80b3-8586eca8e141 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:57:14.922012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd987c8a-65b8-403b-80b3-8586eca8e141 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:57:14.922012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd987c8a-65b8-403b-80b3-8586eca8e141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:57:14.941084 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-bd987c8a-65b8-403b-80b3-8586eca8e141 None None] MFA Rules not processed for user `36dbc938bcd741f2a2b2013b6aa6f008`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:57:14.942808 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bd987c8a-65b8-403b-80b3-8586eca8e141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:57:14.950738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1155/2321] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:57:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:57:18.099973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d6fb2-7edb-4708-ba31-b074c27bb19a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:57:18.114966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d6fb2-7edb-4708-ba31-b074c27bb19a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:57:18.120642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3944ad8-7f55-4c7d-89dd-675393bd4f8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:57:18.121452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3944ad8-7f55-4c7d-89dd-675393bd4f8b service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:57:18.121826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3944ad8-7f55-4c7d-89dd-675393bd4f8b service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:57:18.122171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3944ad8-7f55-4c7d-89dd-675393bd4f8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:57:18.124150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a3944ad8-7f55-4c7d-89dd-675393bd4f8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:57:18.140639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3944ad8-7f55-4c7d-89dd-675393bd4f8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:57:18.142883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3944ad8-7f55-4c7d-89dd-675393bd4f8b service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:57:18.156641 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1167/2322] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:57:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:57:24.236605 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd987c8a-65b8-403b-80b3-8586eca8e141 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:57:24.257208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd987c8a-65b8-403b-80b3-8586eca8e141 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:57:24.262897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083aa420-9d32-443f-b225-9fa0341361cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:57:24.263496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083aa420-9d32-443f-b225-9fa0341361cc service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:57:24.263809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083aa420-9d32-443f-b225-9fa0341361cc service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:57:24.264083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083aa420-9d32-443f-b225-9fa0341361cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:57:24.265776 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-083aa420-9d32-443f-b225-9fa0341361cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:57:24.279569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083aa420-9d32-443f-b225-9fa0341361cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:57:24.281207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083aa420-9d32-443f-b225-9fa0341361cc service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:57:24.291479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1156/2323] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:57:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:57:28.151706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=31 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:57:34.287238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=9 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:57:45.744951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276ad84-6f2f-45b8-ae52-9d5483be5480 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:57:45.745481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276ad84-6f2f-45b8-ae52-9d5483be5480 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:57:45.745481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276ad84-6f2f-45b8-ae52-9d5483be5480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:57:45.766504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-b276ad84-6f2f-45b8-ae52-9d5483be5480 None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:57:45.767628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b276ad84-6f2f-45b8-ae52-9d5483be5480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:57:45.823046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1168/2324] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:57:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:57:45.874132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083aa420-9d32-443f-b225-9fa0341361cc service nova] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:57:45.875880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-083aa420-9d32-443f-b225-9fa0341361cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:57:45.957931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083aa420-9d32-443f-b225-9fa0341361cc service nova] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:57:45.963066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b08847-52a3-4303-b089-1e1d03951ace service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:57:45.963708 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b08847-52a3-4303-b089-1e1d03951ace service swift] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:57:45.963992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b08847-52a3-4303-b089-1e1d03951ace service swift] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:57:45.964262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b08847-52a3-4303-b089-1e1d03951ace service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:57:45.974452 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b08847-52a3-4303-b089-1e1d03951ace service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:57:45.976162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b08847-52a3-4303-b089-1e1d03951ace service swift] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:57:45.985465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1157/2325] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:57:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:57:55.815703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=25 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:57:55.982157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=32 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:05.596050 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276ad84-6f2f-45b8-ae52-9d5483be5480 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:05.598081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b276ad84-6f2f-45b8-ae52-9d5483be5480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:05.653912 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276ad84-6f2f-45b8-ae52-9d5483be5480 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:05.657529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f3911-1e0f-4897-a897-4330e9b99410 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:05.658124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f3911-1e0f-4897-a897-4330e9b99410 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:05.658401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f3911-1e0f-4897-a897-4330e9b99410 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:05.658675 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f3911-1e0f-4897-a897-4330e9b99410 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:05.661039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-fc9f3911-1e0f-4897-a897-4330e9b99410 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:05.671771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f3911-1e0f-4897-a897-4330e9b99410 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:05.673230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f3911-1e0f-4897-a897-4330e9b99410 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:05.681984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1169/2326] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:05 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:09.340532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdca83b-c885-4d9b-b5fe-718944c28bc6 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:09.341039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdca83b-c885-4d9b-b5fe-718944c28bc6 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:09.341146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdca83b-c885-4d9b-b5fe-718944c28bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:09.360803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-5bdca83b-c885-4d9b-b5fe-718944c28bc6 None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:09.362109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5bdca83b-c885-4d9b-b5fe-718944c28bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:09.370226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1158/2327] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:09.382559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f3911-1e0f-4897-a897-4330e9b99410 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:09.396808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f3911-1e0f-4897-a897-4330e9b99410 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:09.402485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30ff57-a8c7-4ca7-aef9-e27edb019acb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:09.403171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ff57-a8c7-4ca7-aef9-e27edb019acb service swift] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:09.403445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ff57-a8c7-4ca7-aef9-e27edb019acb service swift] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:09.403777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ff57-a8c7-4ca7-aef9-e27edb019acb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:09.416429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30ff57-a8c7-4ca7-aef9-e27edb019acb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:09.418363 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30ff57-a8c7-4ca7-aef9-e27edb019acb service swift] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:09.427585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1170/2328] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:58:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:58:19.355448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:19.423934 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=32 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:25.827894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1e0b9-2e93-4350-9ea7-544abb206e70 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:25.828399 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1e0b9-2e93-4350-9ea7-544abb206e70 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:25.828502 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1e0b9-2e93-4350-9ea7-544abb206e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:25.850211 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-b1d1e0b9-2e93-4350-9ea7-544abb206e70 None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:25.852139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b1d1e0b9-2e93-4350-9ea7-544abb206e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:25.867535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1159/2329] 10.222.0.36 () {58 vars in 967 bytes} [Thu Aug 18 14:58:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:25.883327 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30ff57-a8c7-4ca7-aef9-e27edb019acb service swift] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:25.901293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30ff57-a8c7-4ca7-aef9-e27edb019acb service swift] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:25.906864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2d48c-f541-4670-9004-ba0c2900e4f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:25.907476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d48c-f541-4670-9004-ba0c2900e4f1 service swift] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:25.907780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d48c-f541-4670-9004-ba0c2900e4f1 service swift] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:25.908056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d48c-f541-4670-9004-ba0c2900e4f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:25.916888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2d48c-f541-4670-9004-ba0c2900e4f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:25.923742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2d48c-f541-4670-9004-ba0c2900e4f1 service swift] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:25.932645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1171/2330] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:58:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:58:26.975303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1e0b9-2e93-4350-9ea7-544abb206e70 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:26.977235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b1d1e0b9-2e93-4350-9ea7-544abb206e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:26.990771 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1e0b9-2e93-4350-9ea7-544abb206e70 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:26.995945 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065395e6-56ca-4047-890f-d1dade93fd16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:26.996465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065395e6-56ca-4047-890f-d1dade93fd16 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:26.996748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065395e6-56ca-4047-890f-d1dade93fd16 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:26.997308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065395e6-56ca-4047-890f-d1dade93fd16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:26.999159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-065395e6-56ca-4047-890f-d1dade93fd16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:27.071898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065395e6-56ca-4047-890f-d1dade93fd16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:27.073344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065395e6-56ca-4047-890f-d1dade93fd16 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:27.083549 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1160/2331] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:30.658030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2d48c-f541-4670-9004-ba0c2900e4f1 service swift] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:30.659475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-1ef2d48c-f541-4670-9004-ba0c2900e4f1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:30.718444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065395e6-56ca-4047-890f-d1dade93fd16 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:30.725289 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-065395e6-56ca-4047-890f-d1dade93fd16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:30.743037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065395e6-56ca-4047-890f-d1dade93fd16 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:30.747757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2bc36-fa15-49ac-875a-51d0cb588567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:30.748598 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2bc36-fa15-49ac-875a-51d0cb588567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:30.748941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2bc36-fa15-49ac-875a-51d0cb588567 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:30.749304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2bc36-fa15-49ac-875a-51d0cb588567 admin admin] PATH_INFO: `/v3/users/26eaa7f102bd41028983458ae48ff992` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:30.750498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2bc36-fa15-49ac-875a-51d0cb588567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26eaa7f102bd41028983458ae48ff992)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:30.752518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2bc36-fa15-49ac-875a-51d0cb588567 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:30.755441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2d48c-f541-4670-9004-ba0c2900e4f1 service swift] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:30.760847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260af49-be7e-4eec-b077-d330a6b01ab3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:30.761450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260af49-be7e-4eec-b077-d330a6b01ab3 service placement] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:30.761733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260af49-be7e-4eec-b077-d330a6b01ab3 service placement] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:30.762007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260af49-be7e-4eec-b077-d330a6b01ab3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:30.766559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f260af49-be7e-4eec-b077-d330a6b01ab3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:30.782535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260af49-be7e-4eec-b077-d330a6b01ab3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:30.784825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260af49-be7e-4eec-b077-d330a6b01ab3 service placement] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:30.796962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1172/2332] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:30.812870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e9e2bc36-fa15-49ac-875a-51d0cb588567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26eaa7f102bd41028983458ae48ff992'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:30.821338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e9e2bc36-fa15-49ac-875a-51d0cb588567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26eaa7f102bd41028983458ae48ff992'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:30.822040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e9e2bc36-fa15-49ac-875a-51d0cb588567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26eaa7f102bd41028983458ae48ff992'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:30.822652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e9e2bc36-fa15-49ac-875a-51d0cb588567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26eaa7f102bd41028983458ae48ff992'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:30.838236 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-e9e2bc36-fa15-49ac-875a-51d0cb588567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26eaa7f102bd41028983458ae48ff992'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:30.853553 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1161/2333] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:30 2022] DELETE /identity/v3/users/26eaa7f102bd41028983458ae48ff992 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:31.032084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260af49-be7e-4eec-b077-d330a6b01ab3 service placement] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:31.043986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260af49-be7e-4eec-b077-d330a6b01ab3 service placement] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:31.102827 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:31.103454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:31.103753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:31.104024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5 admin admin] PATH_INFO: `/v3/projects/acc902aa109c4d7291c0bab62f80048e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:31.105974 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc902aa109c4d7291c0bab62f80048e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:31.108132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:31.156435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc902aa109c4d7291c0bab62f80048e', 'request_id': 'req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:31.161914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc902aa109c4d7291c0bab62f80048e', 'request_id': 'req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:31.169285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc902aa109c4d7291c0bab62f80048e', 'request_id': 'req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:31.180482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5 admin admin] The token cache is being invalidate because project acc902aa109c4d7291c0bab62f80048e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:31.180774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:31.181477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4130e1c-c57e-4dc8-8536-ba52f0b072c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:31.182950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1173/2334] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:31 2022] DELETE /identity/v3/projects/acc902aa109c4d7291c0bab62f80048e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:31.187931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cec28-5819-4dec-9ead-ba69d63c8bab None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:31.188218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cec28-5819-4dec-9ead-ba69d63c8bab None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:31.188496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cec28-5819-4dec-9ead-ba69d63c8bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:31.210414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-134cec28-5819-4dec-9ead-ba69d63c8bab None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:31.211673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-134cec28-5819-4dec-9ead-ba69d63c8bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:31.261461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:31.272637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1162/2335] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:31.280420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55b9cf-6e31-48e5-80b7-d26b2264ee79 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:31.280648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55b9cf-6e31-48e5-80b7-d26b2264ee79 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:31.280926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55b9cf-6e31-48e5-80b7-d26b2264ee79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:31.295975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:31.302218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-6b55b9cf-6e31-48e5-80b7-d26b2264ee79 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:31.303910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6b55b9cf-6e31-48e5-80b7-d26b2264ee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:31.312582 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1174/2336] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:31.316867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cec28-5819-4dec-9ead-ba69d63c8bab None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:31.326355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cec28-5819-4dec-9ead-ba69d63c8bab None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:31.330919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83abf0b-7347-480a-84db-dd334c164505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:31.331507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83abf0b-7347-480a-84db-dd334c164505 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:31.331831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83abf0b-7347-480a-84db-dd334c164505 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:31.332108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83abf0b-7347-480a-84db-dd334c164505 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:31.332749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83abf0b-7347-480a-84db-dd334c164505 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:58:31.333052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83abf0b-7347-480a-84db-dd334c164505 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:31.334644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83abf0b-7347-480a-84db-dd334c164505 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:31.346695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1163/2337] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:58:31.351211 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55b9cf-6e31-48e5-80b7-d26b2264ee79 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:31.365528 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55b9cf-6e31-48e5-80b7-d26b2264ee79 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:31.370342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cb8d6-0334-4cfe-9ddf-371addd62e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:31.371017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cb8d6-0334-4cfe-9ddf-371addd62e29 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:31.371381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cb8d6-0334-4cfe-9ddf-371addd62e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:31.371740 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cb8d6-0334-4cfe-9ddf-371addd62e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:31.372458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cb8d6-0334-4cfe-9ddf-371addd62e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:31.374440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cb8d6-0334-4cfe-9ddf-371addd62e29 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:31.406254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1175/2338] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:31 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:31.410501 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83abf0b-7347-480a-84db-dd334c164505 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:31.423295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83abf0b-7347-480a-84db-dd334c164505 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:31.479919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222470e-7194-49bd-b865-6e851faf3808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:31.480785 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222470e-7194-49bd-b865-6e851faf3808 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:31.480785 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222470e-7194-49bd-b865-6e851faf3808 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:31.481035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222470e-7194-49bd-b865-6e851faf3808 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:31.481763 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222470e-7194-49bd-b865-6e851faf3808 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:31.485219 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222470e-7194-49bd-b865-6e851faf3808 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:31.516096 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1164/2339] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:31 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:58:31.521818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cb8d6-0334-4cfe-9ddf-371addd62e29 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:31.532310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cb8d6-0334-4cfe-9ddf-371addd62e29 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:31.537876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6811460-8ed8-4dad-8ba7-485a6832879e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:31.538488 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6811460-8ed8-4dad-8ba7-485a6832879e admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:31.538763 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6811460-8ed8-4dad-8ba7-485a6832879e admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:31.539038 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6811460-8ed8-4dad-8ba7-485a6832879e admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:31.539766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6811460-8ed8-4dad-8ba7-485a6832879e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:31.541360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6811460-8ed8-4dad-8ba7-485a6832879e admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:31.552392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1176/2340] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:31.556207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222470e-7194-49bd-b865-6e851faf3808 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:31.566100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222470e-7194-49bd-b865-6e851faf3808 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:31.571330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d45ce-23f2-49b7-86d6-f75215c3953f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:31.571975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d45ce-23f2-49b7-86d6-f75215c3953f admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:31.572255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d45ce-23f2-49b7-86d6-f75215c3953f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:31.572528 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d45ce-23f2-49b7-86d6-f75215c3953f admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:31.573180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d45ce-23f2-49b7-86d6-f75215c3953f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:31.574770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d45ce-23f2-49b7-86d6-f75215c3953f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:31.586113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1165/2341] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:31.590393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6811460-8ed8-4dad-8ba7-485a6832879e admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:31.601099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6811460-8ed8-4dad-8ba7-485a6832879e admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:31.605815 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34def6b-817a-4ee7-8962-1f43a12dc9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:31.606462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34def6b-817a-4ee7-8962-1f43a12dc9ef admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:31.606738 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34def6b-817a-4ee7-8962-1f43a12dc9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:31.607007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34def6b-817a-4ee7-8962-1f43a12dc9ef admin admin] PATH_INFO: `/v3/projects/ecaae864012243ecb812d5d71174e1a7/users/331b75a7abac43a7a00aa29f94a19c63/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:31.622595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34def6b-817a-4ee7-8962-1f43a12dc9ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecaae864012243ecb812d5d71174e1a7, user_id=331b75a7abac43a7a00aa29f94a19c63, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:31.625273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34def6b-817a-4ee7-8962-1f43a12dc9ef admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:31.648886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1177/2342] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:31 2022] PUT /identity/v3/projects/ecaae864012243ecb812d5d71174e1a7/users/331b75a7abac43a7a00aa29f94a19c63/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:31.658773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35a99a-e9c4-4f02-a429-c3879d9d3c07 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:31.659056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35a99a-e9c4-4f02-a429-c3879d9d3c07 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:31.659333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35a99a-e9c4-4f02-a429-c3879d9d3c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:31.689081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-4f35a99a-e9c4-4f02-a429-c3879d9d3c07 None None] MFA Rules not processed for user `331b75a7abac43a7a00aa29f94a19c63`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:31.690308 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4f35a99a-e9c4-4f02-a429-c3879d9d3c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:31.758335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1166/2343] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:31.858186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34def6b-817a-4ee7-8962-1f43a12dc9ef admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:31.860075 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e34def6b-817a-4ee7-8962-1f43a12dc9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:31.940138 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34def6b-817a-4ee7-8962-1f43a12dc9ef admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:31.946571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41b262-03b7-4d23-a8b3-a4590d53a112 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:31.947180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41b262-03b7-4d23-a8b3-a4590d53a112 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:31.947456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41b262-03b7-4d23-a8b3-a4590d53a112 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:31.947768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41b262-03b7-4d23-a8b3-a4590d53a112 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:31.955828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41b262-03b7-4d23-a8b3-a4590d53a112 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:31.957426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41b262-03b7-4d23-a8b3-a4590d53a112 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:31.966947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1178/2344] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:34.254313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35a99a-e9c4-4f02-a429-c3879d9d3c07 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:34.355900 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:34.364740 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35a99a-e9c4-4f02-a429-c3879d9d3c07 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:34.378337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:34.379111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:34.379508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:34.379930 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] PATH_INFO: `/v3/users/331b75a7abac43a7a00aa29f94a19c63` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:34.381284 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] RBAC: Authorizing `identity:delete_user(user_id=331b75a7abac43a7a00aa29f94a19c63)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:34.383273 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:34.450522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '331b75a7abac43a7a00aa29f94a19c63'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:34.465969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '331b75a7abac43a7a00aa29f94a19c63'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:34.467001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '331b75a7abac43a7a00aa29f94a19c63'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:34.467981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '331b75a7abac43a7a00aa29f94a19c63'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:34.484162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '331b75a7abac43a7a00aa29f94a19c63'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:34.501469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1167/2345] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:34 2022] DELETE /identity/v3/users/331b75a7abac43a7a00aa29f94a19c63 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:34.506068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc1c1e-8901-4dae-bf73-f2a3645f0d70 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:34.507769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc1c1e-8901-4dae-bf73-f2a3645f0d70 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:34.508224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc1c1e-8901-4dae-bf73-f2a3645f0d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:34.531336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-52dc1c1e-8901-4dae-bf73-f2a3645f0d70 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:34.532638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-52dc1c1e-8901-4dae-bf73-f2a3645f0d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:34.615739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1179/2346] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:34.626317 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:34.628088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:34.697475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2464436-d03b-4fd3-a7e4-a3d29ab4886c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:34.701294 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48730d-4fd1-4e1d-9937-bcc1ea25340e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:34.701997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48730d-4fd1-4e1d-9937-bcc1ea25340e service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:34.702341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48730d-4fd1-4e1d-9937-bcc1ea25340e service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:34.702717 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48730d-4fd1-4e1d-9937-bcc1ea25340e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:34.714244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48730d-4fd1-4e1d-9937-bcc1ea25340e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:34.715960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48730d-4fd1-4e1d-9937-bcc1ea25340e service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:34.726644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1168/2347] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:35.036459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc1c1e-8901-4dae-bf73-f2a3645f0d70 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:35.038083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-52dc1c1e-8901-4dae-bf73-f2a3645f0d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:35.055922 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc1c1e-8901-4dae-bf73-f2a3645f0d70 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:35.061466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194a1e0-c2b5-4511-8d18-c864a9f3437e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:35.062356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194a1e0-c2b5-4511-8d18-c864a9f3437e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:35.062704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194a1e0-c2b5-4511-8d18-c864a9f3437e admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:35.063036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194a1e0-c2b5-4511-8d18-c864a9f3437e admin admin] PATH_INFO: `/v3/projects/ecaae864012243ecb812d5d71174e1a7` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:35.072001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194a1e0-c2b5-4511-8d18-c864a9f3437e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecaae864012243ecb812d5d71174e1a7)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:35.074518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194a1e0-c2b5-4511-8d18-c864a9f3437e admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:35.105018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:35.128242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8194a1e0-c2b5-4511-8d18-c864a9f3437e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecaae864012243ecb812d5d71174e1a7', 'request_id': 'req-8194a1e0-c2b5-4511-8d18-c864a9f3437e'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:35.134525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8194a1e0-c2b5-4511-8d18-c864a9f3437e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecaae864012243ecb812d5d71174e1a7', 'request_id': 'req-8194a1e0-c2b5-4511-8d18-c864a9f3437e'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:35.144199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8194a1e0-c2b5-4511-8d18-c864a9f3437e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecaae864012243ecb812d5d71174e1a7', 'request_id': 'req-8194a1e0-c2b5-4511-8d18-c864a9f3437e'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:35.156345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8194a1e0-c2b5-4511-8d18-c864a9f3437e admin admin] The token cache is being invalidate because project ecaae864012243ecb812d5d71174e1a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:35.156630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8194a1e0-c2b5-4511-8d18-c864a9f3437e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:35.157278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8194a1e0-c2b5-4511-8d18-c864a9f3437e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:35.158791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1180/2348] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:35 2022] DELETE /identity/v3/projects/ecaae864012243ecb812d5d71174e1a7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:35.163742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a45b46-132e-4ec9-a7b5-c46e490a6bb9 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:35.164150 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a45b46-132e-4ec9-a7b5-c46e490a6bb9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:35.164494 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a45b46-132e-4ec9-a7b5-c46e490a6bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:35.184327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-24a45b46-132e-4ec9-a7b5-c46e490a6bb9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:35.185561 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-24a45b46-132e-4ec9-a7b5-c46e490a6bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:35.257243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1169/2349] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:35.265678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50e5ba-8fc7-4e23-af59-747fb71c3832 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:35.266813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50e5ba-8fc7-4e23-af59-747fb71c3832 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:35.266813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50e5ba-8fc7-4e23-af59-747fb71c3832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:35.290192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-2f50e5ba-8fc7-4e23-af59-747fb71c3832 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:35.290446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2f50e5ba-8fc7-4e23-af59-747fb71c3832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:35.298838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1181/2350] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:35.303320 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a45b46-132e-4ec9-a7b5-c46e490a6bb9 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:35.314562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a45b46-132e-4ec9-a7b5-c46e490a6bb9 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:35.319555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac9ebe-9e4d-4511-8f1b-699281ca95c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:35.320392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac9ebe-9e4d-4511-8f1b-699281ca95c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:35.320675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac9ebe-9e4d-4511-8f1b-699281ca95c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:35.320937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac9ebe-9e4d-4511-8f1b-699281ca95c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:35.321754 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac9ebe-9e4d-4511-8f1b-699281ca95c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:58:35.322209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac9ebe-9e4d-4511-8f1b-699281ca95c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:35.324008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac9ebe-9e4d-4511-8f1b-699281ca95c7 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:35.338291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1170/2351] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:58:35.342262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50e5ba-8fc7-4e23-af59-747fb71c3832 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:35.355060 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50e5ba-8fc7-4e23-af59-747fb71c3832 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:35.361664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71718866-a696-401c-8f99-539ec627971e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:35.362389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71718866-a696-401c-8f99-539ec627971e admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:35.362770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71718866-a696-401c-8f99-539ec627971e admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:35.363146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71718866-a696-401c-8f99-539ec627971e admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:35.363956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71718866-a696-401c-8f99-539ec627971e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:35.366015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71718866-a696-401c-8f99-539ec627971e admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:35.402651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1182/2352] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:35 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:58:35.406560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac9ebe-9e4d-4511-8f1b-699281ca95c7 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:35.413668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:35.417479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac9ebe-9e4d-4511-8f1b-699281ca95c7 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:35.494619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ced5-aec1-4741-b6dd-c30afb3ee5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:35.495257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ced5-aec1-4741-b6dd-c30afb3ee5df admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:35.495543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ced5-aec1-4741-b6dd-c30afb3ee5df admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:35.495857 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ced5-aec1-4741-b6dd-c30afb3ee5df admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:35.496569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3ced5-aec1-4741-b6dd-c30afb3ee5df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:35.498899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3ced5-aec1-4741-b6dd-c30afb3ee5df admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:35.525765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1171/2353] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:35 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:58:35.530369 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71718866-a696-401c-8f99-539ec627971e admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:35.546543 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71718866-a696-401c-8f99-539ec627971e admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:35.554217 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390a219-4514-400e-9bad-4fc548974d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:35.557258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390a219-4514-400e-9bad-4fc548974d35 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:35.557655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390a219-4514-400e-9bad-4fc548974d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:35.558064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390a219-4514-400e-9bad-4fc548974d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:35.558822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390a219-4514-400e-9bad-4fc548974d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:35.562005 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390a219-4514-400e-9bad-4fc548974d35 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:35.583181 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1183/2354] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:35.584904 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ced5-aec1-4741-b6dd-c30afb3ee5df admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:35.597117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ced5-aec1-4741-b6dd-c30afb3ee5df admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:35.603338 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155da87-7499-4398-9f45-97e888c232c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:35.604100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155da87-7499-4398-9f45-97e888c232c0 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:35.604455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155da87-7499-4398-9f45-97e888c232c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:35.604805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155da87-7499-4398-9f45-97e888c232c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:35.605536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155da87-7499-4398-9f45-97e888c232c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:35.607739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155da87-7499-4398-9f45-97e888c232c0 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:35.623766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1172/2355] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:35.626146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390a219-4514-400e-9bad-4fc548974d35 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:35.640418 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390a219-4514-400e-9bad-4fc548974d35 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:35.647199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9060602-b2e9-4370-aa62-dd2c6e4255c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:35.647925 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9060602-b2e9-4370-aa62-dd2c6e4255c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:35.651143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9060602-b2e9-4370-aa62-dd2c6e4255c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:35.651143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9060602-b2e9-4370-aa62-dd2c6e4255c6 admin admin] PATH_INFO: `/v3/projects/27aaaf73d60b41f088318760f1490e76/users/c0a5d83f672d4ecdad08e3057bfea6af/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:35.671365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9060602-b2e9-4370-aa62-dd2c6e4255c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27aaaf73d60b41f088318760f1490e76, user_id=c0a5d83f672d4ecdad08e3057bfea6af, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:35.674392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9060602-b2e9-4370-aa62-dd2c6e4255c6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:35.696951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1184/2356] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:35 2022] PUT /identity/v3/projects/27aaaf73d60b41f088318760f1490e76/users/c0a5d83f672d4ecdad08e3057bfea6af/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:35.699065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37ea2d-65ad-4542-ba44-182fc86b9818 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:35.699353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37ea2d-65ad-4542-ba44-182fc86b9818 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:35.699622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37ea2d-65ad-4542-ba44-182fc86b9818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:35.734549 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-cc37ea2d-65ad-4542-ba44-182fc86b9818 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:35.735737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-cc37ea2d-65ad-4542-ba44-182fc86b9818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:35.799235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1173/2357] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:35.810260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9060602-b2e9-4370-aa62-dd2c6e4255c6 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:35.812330 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c9060602-b2e9-4370-aa62-dd2c6e4255c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:35.873966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9060602-b2e9-4370-aa62-dd2c6e4255c6 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:35.878036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f8d69-91d4-4e97-9b43-faca4ee4058b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:35.878681 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f8d69-91d4-4e97-9b43-faca4ee4058b service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:35.878961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f8d69-91d4-4e97-9b43-faca4ee4058b service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:35.879234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f8d69-91d4-4e97-9b43-faca4ee4058b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:35.889461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f8d69-91d4-4e97-9b43-faca4ee4058b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:35.890942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f8d69-91d4-4e97-9b43-faca4ee4058b service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:35.904128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1185/2358] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:36.578018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27c38f-a7c3-4330-b116-b517ed1b42b9 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:36.578570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27c38f-a7c3-4330-b116-b517ed1b42b9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:36.578570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27c38f-a7c3-4330-b116-b517ed1b42b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:36.608783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-5f27c38f-a7c3-4330-b116-b517ed1b42b9 None None] MFA Rules not processed for user `c0a5d83f672d4ecdad08e3057bfea6af`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:36.610126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5f27c38f-a7c3-4330-b116-b517ed1b42b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:36.685335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1174/2359] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:36.689823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f8d69-91d4-4e97-9b43-faca4ee4058b service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:36.701040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f8d69-91d4-4e97-9b43-faca4ee4058b service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:36.705754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3544cd5-466d-4dfe-9b6d-b7eeab425449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:36.706432 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3544cd5-466d-4dfe-9b6d-b7eeab425449 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:36.706768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3544cd5-466d-4dfe-9b6d-b7eeab425449 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:36.707125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3544cd5-466d-4dfe-9b6d-b7eeab425449 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:36.708581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3544cd5-466d-4dfe-9b6d-b7eeab425449 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:36.710629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3544cd5-466d-4dfe-9b6d-b7eeab425449 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:36.743725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1186/2360] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:36 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:58:36.747237 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27c38f-a7c3-4330-b116-b517ed1b42b9 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:36.761214 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27c38f-a7c3-4330-b116-b517ed1b42b9 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:36.784652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:36.850041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885979c-efb3-4966-bdfe-0779f4a54a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:36.850784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885979c-efb3-4966-bdfe-0779f4a54a73 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:36.851161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885979c-efb3-4966-bdfe-0779f4a54a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:36.851522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885979c-efb3-4966-bdfe-0779f4a54a73 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:36.852326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885979c-efb3-4966-bdfe-0779f4a54a73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:36.854410 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885979c-efb3-4966-bdfe-0779f4a54a73 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:36.880954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1175/2361] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:36 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:36.884780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3544cd5-466d-4dfe-9b6d-b7eeab425449 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:36.897024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3544cd5-466d-4dfe-9b6d-b7eeab425449 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:36.903992 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209edf7-fdde-4b2f-81d0-1fc7ba09f477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:36.904862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209edf7-fdde-4b2f-81d0-1fc7ba09f477 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:36.905338 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209edf7-fdde-4b2f-81d0-1fc7ba09f477 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:36.905684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209edf7-fdde-4b2f-81d0-1fc7ba09f477 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:36.906424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209edf7-fdde-4b2f-81d0-1fc7ba09f477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:36.908280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209edf7-fdde-4b2f-81d0-1fc7ba09f477 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:36.920152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1187/2362] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:36.924708 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885979c-efb3-4966-bdfe-0779f4a54a73 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:36.935180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885979c-efb3-4966-bdfe-0779f4a54a73 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:36.941004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daac070-f4f5-4e1d-bba9-6178630cb6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:36.941678 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daac070-f4f5-4e1d-bba9-6178630cb6d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:36.941967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daac070-f4f5-4e1d-bba9-6178630cb6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:36.942300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daac070-f4f5-4e1d-bba9-6178630cb6d4 admin admin] PATH_INFO: `/v3/projects/04f138fe94424d97a732140e20c61d90/users/91129c3447504c119489b0b740d3fc66/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:36.959004 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daac070-f4f5-4e1d-bba9-6178630cb6d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f138fe94424d97a732140e20c61d90, user_id=91129c3447504c119489b0b740d3fc66, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:36.961724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daac070-f4f5-4e1d-bba9-6178630cb6d4 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:36.975613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1176/2363] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:36 2022] PUT /identity/v3/projects/04f138fe94424d97a732140e20c61d90/users/91129c3447504c119489b0b740d3fc66/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:36.979801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209edf7-fdde-4b2f-81d0-1fc7ba09f477 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:36.989837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209edf7-fdde-4b2f-81d0-1fc7ba09f477 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:37.001656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:37.047250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a904cc9-42ad-4b99-8d7f-31e5f1fb7ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:37.047882 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a904cc9-42ad-4b99-8d7f-31e5f1fb7ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:37.048163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a904cc9-42ad-4b99-8d7f-31e5f1fb7ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:37.048433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a904cc9-42ad-4b99-8d7f-31e5f1fb7ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:37.049070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a904cc9-42ad-4b99-8d7f-31e5f1fb7ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:37.050663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a904cc9-42ad-4b99-8d7f-31e5f1fb7ce9 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:37.060804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1188/2364] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:37.064709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daac070-f4f5-4e1d-bba9-6178630cb6d4 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:37.074987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daac070-f4f5-4e1d-bba9-6178630cb6d4 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:37.079480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd2e8a-d99d-4a19-bc05-fbbb83592c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:37.080191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2e8a-d99d-4a19-bc05-fbbb83592c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:37.080470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2e8a-d99d-4a19-bc05-fbbb83592c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:37.080741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2e8a-d99d-4a19-bc05-fbbb83592c08 admin admin] PATH_INFO: `/v3/projects/04f138fe94424d97a732140e20c61d90/users/91129c3447504c119489b0b740d3fc66/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:37.082628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd2e8a-d99d-4a19-bc05-fbbb83592c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f138fe94424d97a732140e20c61d90, user_id=91129c3447504c119489b0b740d3fc66, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:37.085780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd2e8a-d99d-4a19-bc05-fbbb83592c08 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:37.095988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1177/2365] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:37 2022] PUT /identity/v3/projects/04f138fe94424d97a732140e20c61d90/users/91129c3447504c119489b0b740d3fc66/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:37.774654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16d64e-7392-4b52-bde2-93f1da3eeb0c None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:37.775651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16d64e-7392-4b52-bde2-93f1da3eeb0c None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:37.775954 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16d64e-7392-4b52-bde2-93f1da3eeb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:37.804988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-dc16d64e-7392-4b52-bde2-93f1da3eeb0c None None] MFA Rules not processed for user `91129c3447504c119489b0b740d3fc66`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:37.806486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-dc16d64e-7392-4b52-bde2-93f1da3eeb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:37.878970 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1189/2366] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:37.891130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd2e8a-d99d-4a19-bc05-fbbb83592c08 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:37.892938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-32cd2e8a-d99d-4a19-bc05-fbbb83592c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:37.959598 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd2e8a-d99d-4a19-bc05-fbbb83592c08 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:37.963363 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af0ec3-7b0f-4ab3-8c53-3edb5e9466d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:37.963996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af0ec3-7b0f-4ab3-8c53-3edb5e9466d3 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:37.964272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af0ec3-7b0f-4ab3-8c53-3edb5e9466d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:37.964540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af0ec3-7b0f-4ab3-8c53-3edb5e9466d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:38.026977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af0ec3-7b0f-4ab3-8c53-3edb5e9466d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:38.028429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af0ec3-7b0f-4ab3-8c53-3edb5e9466d3 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:38.037892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1178/2367] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:39.336607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16d64e-7392-4b52-bde2-93f1da3eeb0c None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:39.338029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-dc16d64e-7392-4b52-bde2-93f1da3eeb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:39.399944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16d64e-7392-4b52-bde2-93f1da3eeb0c None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:39.403482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9bc6e-d121-47bf-9a4e-6cae66defb1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:39.404107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9bc6e-d121-47bf-9a4e-6cae66defb1b service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:39.404386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9bc6e-d121-47bf-9a4e-6cae66defb1b service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:39.404659 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9bc6e-d121-47bf-9a4e-6cae66defb1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:39.406378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8bb9bc6e-d121-47bf-9a4e-6cae66defb1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:39.413827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af0ec3-7b0f-4ab3-8c53-3edb5e9466d3 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:39.415243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-61af0ec3-7b0f-4ab3-8c53-3edb5e9466d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:39.426929 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af0ec3-7b0f-4ab3-8c53-3edb5e9466d3 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:39.430817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5b37f-8bb9-4c40-a837-164b08d706db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:39.431434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5b37f-8bb9-4c40-a837-164b08d706db service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:39.431743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5b37f-8bb9-4c40-a837-164b08d706db service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:39.432041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5b37f-8bb9-4c40-a837-164b08d706db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:39.441108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5b37f-8bb9-4c40-a837-164b08d706db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:39.443022 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5b37f-8bb9-4c40-a837-164b08d706db service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:39.453015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1179/2368] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:39.463840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:39.476691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9bc6e-d121-47bf-9a4e-6cae66defb1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:39.478130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9bc6e-d121-47bf-9a4e-6cae66defb1b service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:39.490152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1190/2369] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:39.802408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5b37f-8bb9-4c40-a837-164b08d706db service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:39.811549 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5b37f-8bb9-4c40-a837-164b08d706db service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:39.815462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb9fd4-2f7f-4225-8954-8d85f4f1f540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:39.816368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb9fd4-2f7f-4225-8954-8d85f4f1f540 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:39.816876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb9fd4-2f7f-4225-8954-8d85f4f1f540 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:39.816876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb9fd4-2f7f-4225-8954-8d85f4f1f540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:39.819742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-90eb9fd4-2f7f-4225-8954-8d85f4f1f540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:39.828752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:39.833050 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb9fd4-2f7f-4225-8954-8d85f4f1f540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:39.835061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb9fd4-2f7f-4225-8954-8d85f4f1f540 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:39.845734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1180/2370] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:43.517928 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9bc6e-d121-47bf-9a4e-6cae66defb1b service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:43.519975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8bb9bc6e-d121-47bf-9a4e-6cae66defb1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:43.541689 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9bc6e-d121-47bf-9a4e-6cae66defb1b service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:43.560579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:43.561496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:43.561926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:43.562425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] PATH_INFO: `/v3/users/36dbc938bcd741f2a2b2013b6aa6f008` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:43.564080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36dbc938bcd741f2a2b2013b6aa6f008)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:43.568264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:43.691111 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36dbc938bcd741f2a2b2013b6aa6f008'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:43.704116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36dbc938bcd741f2a2b2013b6aa6f008'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:43.705658 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36dbc938bcd741f2a2b2013b6aa6f008'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:43.725202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36dbc938bcd741f2a2b2013b6aa6f008'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:43.735966 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb9fd4-2f7f-4225-8954-8d85f4f1f540 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:43.737533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-90eb9fd4-2f7f-4225-8954-8d85f4f1f540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:43.737950 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36dbc938bcd741f2a2b2013b6aa6f008'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:43.752655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1191/2371] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:43 2022] DELETE /identity/v3/users/36dbc938bcd741f2a2b2013b6aa6f008 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:43.766539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb9fd4-2f7f-4225-8954-8d85f4f1f540 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:43.851143 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220149df-26e6-4d47-97e6-5ff187d9064c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:43.851907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220149df-26e6-4d47-97e6-5ff187d9064c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:43.852475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220149df-26e6-4d47-97e6-5ff187d9064c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:43.852956 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220149df-26e6-4d47-97e6-5ff187d9064c admin admin] PATH_INFO: `/v3/users/4d96208474754b2381d769d807689bbb` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:43.854692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220149df-26e6-4d47-97e6-5ff187d9064c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d96208474754b2381d769d807689bbb)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:43.857357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220149df-26e6-4d47-97e6-5ff187d9064c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:43.946229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-220149df-26e6-4d47-97e6-5ff187d9064c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d96208474754b2381d769d807689bbb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:43.964159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-220149df-26e6-4d47-97e6-5ff187d9064c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d96208474754b2381d769d807689bbb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:43.966311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-220149df-26e6-4d47-97e6-5ff187d9064c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d96208474754b2381d769d807689bbb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:43.966311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-220149df-26e6-4d47-97e6-5ff187d9064c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d96208474754b2381d769d807689bbb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:43.973362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:43.974973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:43.989891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6432c9-3df0-450e-9c66-1d039086da86 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:43.992656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-220149df-26e6-4d47-97e6-5ff187d9064c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d96208474754b2381d769d807689bbb'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:43.996679 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7759529-e8b9-4c6a-99be-933c1c2d2e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:43.997941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759529-e8b9-4c6a-99be-933c1c2d2e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:43.998816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759529-e8b9-4c6a-99be-933c1c2d2e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:43.999469 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759529-e8b9-4c6a-99be-933c1c2d2e75 admin admin] PATH_INFO: `/v3/projects/55c9fe0cbbfa47bca81668c039afc092` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.001927 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7759529-e8b9-4c6a-99be-933c1c2d2e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c9fe0cbbfa47bca81668c039afc092)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.006083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7759529-e8b9-4c6a-99be-933c1c2d2e75 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.012214 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1181/2372] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:43 2022] DELETE /identity/v3/users/4d96208474754b2381d769d807689bbb => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:44.064747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e7759529-e8b9-4c6a-99be-933c1c2d2e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c9fe0cbbfa47bca81668c039afc092', 'request_id': 'req-e7759529-e8b9-4c6a-99be-933c1c2d2e75'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:44.072574 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e7759529-e8b9-4c6a-99be-933c1c2d2e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c9fe0cbbfa47bca81668c039afc092', 'request_id': 'req-e7759529-e8b9-4c6a-99be-933c1c2d2e75'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:44.081569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e7759529-e8b9-4c6a-99be-933c1c2d2e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c9fe0cbbfa47bca81668c039afc092', 'request_id': 'req-e7759529-e8b9-4c6a-99be-933c1c2d2e75'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:44.094505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e7759529-e8b9-4c6a-99be-933c1c2d2e75 admin admin] The token cache is being invalidate because project 55c9fe0cbbfa47bca81668c039afc092 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:44.094877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e7759529-e8b9-4c6a-99be-933c1c2d2e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:44.096091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e7759529-e8b9-4c6a-99be-933c1c2d2e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:44.097995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1192/2373] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:43 2022] DELETE /identity/v3/projects/55c9fe0cbbfa47bca81668c039afc092 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:44.102846 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278594c0-9a71-4864-9e0d-069a41af9035 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.103130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278594c0-9a71-4864-9e0d-069a41af9035 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.103405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278594c0-9a71-4864-9e0d-069a41af9035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.123544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-278594c0-9a71-4864-9e0d-069a41af9035 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:44.124798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-278594c0-9a71-4864-9e0d-069a41af9035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:44.187208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1182/2374] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:44.195736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7d6b4-9f45-4843-be21-43d3a1251bd7 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.195999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7d6b4-9f45-4843-be21-43d3a1251bd7 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.196271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7d6b4-9f45-4843-be21-43d3a1251bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.199091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278594c0-9a71-4864-9e0d-069a41af9035 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.200488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-278594c0-9a71-4864-9e0d-069a41af9035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:44.216594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278594c0-9a71-4864-9e0d-069a41af9035 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.217298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-44c7d6b4-9f45-4843-be21-43d3a1251bd7 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:44.218529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-44c7d6b4-9f45-4843-be21-43d3a1251bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:44.221368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.222720 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.223010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.223417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e admin admin] PATH_INFO: `/v3/projects/bde4e338430442449fcc203cd9493bc3` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.224881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde4e338430442449fcc203cd9493bc3)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.226768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.227957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1193/2375] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:44.231802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7d6b4-9f45-4843-be21-43d3a1251bd7 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.236061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:44.242603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7d6b4-9f45-4843-be21-43d3a1251bd7 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.250418 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966d019-68f2-456b-baa9-99fd4d6996f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.251973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966d019-68f2-456b-baa9-99fd4d6996f7 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.253114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966d019-68f2-456b-baa9-99fd4d6996f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.253606 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966d019-68f2-456b-baa9-99fd4d6996f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.254523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966d019-68f2-456b-baa9-99fd4d6996f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:58:44.255354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966d019-68f2-456b-baa9-99fd4d6996f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.257412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966d019-68f2-456b-baa9-99fd4d6996f7 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.270715 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1194/2376] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:58:44.275716 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966d019-68f2-456b-baa9-99fd4d6996f7 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.285490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde4e338430442449fcc203cd9493bc3', 'request_id': 'req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:44.287076 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966d019-68f2-456b-baa9-99fd4d6996f7 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.295067 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde4e338430442449fcc203cd9493bc3', 'request_id': 'req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:44.303788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde4e338430442449fcc203cd9493bc3', 'request_id': 'req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:44.313620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:44.321780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e admin admin] The token cache is being invalidate because project bde4e338430442449fcc203cd9493bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:44.322204 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:44.323083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4bfd7d93-80c3-49ef-968a-68c1cab11a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:44.327988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1183/2377] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:44 2022] DELETE /identity/v3/projects/bde4e338430442449fcc203cd9493bc3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:44.330721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da8be9-24a0-47d9-b599-325c378565b2 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.331027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da8be9-24a0-47d9-b599-325c378565b2 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.331315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da8be9-24a0-47d9-b599-325c378565b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.353623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a0da8be9-24a0-47d9-b599-325c378565b2 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:44.354869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a0da8be9-24a0-47d9-b599-325c378565b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:44.364978 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17762bfe-5a79-44bf-89ed-4a16a7d9f4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.365946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17762bfe-5a79-44bf-89ed-4a16a7d9f4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.366261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17762bfe-5a79-44bf-89ed-4a16a7d9f4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.366530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17762bfe-5a79-44bf-89ed-4a16a7d9f4c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.367415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17762bfe-5a79-44bf-89ed-4a16a7d9f4c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.370084 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17762bfe-5a79-44bf-89ed-4a16a7d9f4c4 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.403733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1195/2378] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:44 2022] POST /identity/v3/projects => generated 403 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:44.408122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17762bfe-5a79-44bf-89ed-4a16a7d9f4c4 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.409863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-17762bfe-5a79-44bf-89ed-4a16a7d9f4c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:44.421579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1184/2379] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:44.430702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc06742-0843-4568-86e8-ee465513c475 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.431049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc06742-0843-4568-86e8-ee465513c475 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.431314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc06742-0843-4568-86e8-ee465513c475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.452359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-afc06742-0843-4568-86e8-ee465513c475 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:44.453595 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-afc06742-0843-4568-86e8-ee465513c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:44.478521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17762bfe-5a79-44bf-89ed-4a16a7d9f4c4 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.483955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f5d71-6a97-4481-8ae1-d6fd7c691f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.484565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5d71-6a97-4481-8ae1-d6fd7c691f0d admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.484828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5d71-6a97-4481-8ae1-d6fd7c691f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.485107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5d71-6a97-4481-8ae1-d6fd7c691f0d admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.485726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f5d71-6a97-4481-8ae1-d6fd7c691f0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.487656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f5d71-6a97-4481-8ae1-d6fd7c691f0d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.511104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1196/2380] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:44 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:44.515890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f5d71-6a97-4481-8ae1-d6fd7c691f0d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.527064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f5d71-6a97-4481-8ae1-d6fd7c691f0d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.532288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb8b73-e296-4a2e-896a-ec6cc68266a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.532932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb8b73-e296-4a2e-896a-ec6cc68266a7 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.533215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb8b73-e296-4a2e-896a-ec6cc68266a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.533496 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb8b73-e296-4a2e-896a-ec6cc68266a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.534328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb8b73-e296-4a2e-896a-ec6cc68266a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.536867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb8b73-e296-4a2e-896a-ec6cc68266a7 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.549070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1197/2381] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:44.553608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb8b73-e296-4a2e-896a-ec6cc68266a7 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.564226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb8b73-e296-4a2e-896a-ec6cc68266a7 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.570183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620675d7-a905-4463-8275-3867c79f07f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.570815 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620675d7-a905-4463-8275-3867c79f07f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.571098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620675d7-a905-4463-8275-3867c79f07f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.571373 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620675d7-a905-4463-8275-3867c79f07f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.572072 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620675d7-a905-4463-8275-3867c79f07f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.573699 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620675d7-a905-4463-8275-3867c79f07f5 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.585959 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1198/2382] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:58:44.589990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620675d7-a905-4463-8275-3867c79f07f5 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.600794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620675d7-a905-4463-8275-3867c79f07f5 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.605600 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a515616-5fb9-4da5-bf4a-7348211d9ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.606273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a515616-5fb9-4da5-bf4a-7348211d9ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.606553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a515616-5fb9-4da5-bf4a-7348211d9ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.606828 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a515616-5fb9-4da5-bf4a-7348211d9ee3 admin admin] PATH_INFO: `/v3/projects/929725e14c9e499ca35214ffd8b8a611/users/a29226e406e844068e7c2f123d0aba7b/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.623140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a515616-5fb9-4da5-bf4a-7348211d9ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=929725e14c9e499ca35214ffd8b8a611, user_id=a29226e406e844068e7c2f123d0aba7b, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.626112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a515616-5fb9-4da5-bf4a-7348211d9ee3 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.640865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1199/2383] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:44 2022] PUT /identity/v3/projects/929725e14c9e499ca35214ffd8b8a611/users/a29226e406e844068e7c2f123d0aba7b/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:44.645966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941f586-cf07-4b6d-b6f6-5992bb98e69f None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.646250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941f586-cf07-4b6d-b6f6-5992bb98e69f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.646517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941f586-cf07-4b6d-b6f6-5992bb98e69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.649768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1185/2384] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:44.653705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc06742-0843-4568-86e8-ee465513c475 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.663844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc06742-0843-4568-86e8-ee465513c475 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.668475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-6941f586-cf07-4b6d-b6f6-5992bb98e69f None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:44.669678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6941f586-cf07-4b6d-b6f6-5992bb98e69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:44.697693 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:44.720032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:44.724723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f5e40-e6c0-4dd1-8c8a-d2ab546464ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.725336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f5e40-e6c0-4dd1-8c8a-d2ab546464ef admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.725611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f5e40-e6c0-4dd1-8c8a-d2ab546464ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.725880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f5e40-e6c0-4dd1-8c8a-d2ab546464ef admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.726656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f5e40-e6c0-4dd1-8c8a-d2ab546464ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:58:44.726959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f5e40-e6c0-4dd1-8c8a-d2ab546464ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.728713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f5e40-e6c0-4dd1-8c8a-d2ab546464ef admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.738520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1200/2385] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:44.745041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1186/2386] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:58:44.746166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941f586-cf07-4b6d-b6f6-5992bb98e69f None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.748588 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f5e40-e6c0-4dd1-8c8a-d2ab546464ef admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.750351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b17f5e40-e6c0-4dd1-8c8a-d2ab546464ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:44.756487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941f586-cf07-4b6d-b6f6-5992bb98e69f None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.761727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e1639-b4c5-499d-813c-f3a6ffe6a836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.762497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e1639-b4c5-499d-813c-f3a6ffe6a836 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.762930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e1639-b4c5-499d-813c-f3a6ffe6a836 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.763638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e1639-b4c5-499d-813c-f3a6ffe6a836 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.764297 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e1639-b4c5-499d-813c-f3a6ffe6a836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.766186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e1639-b4c5-499d-813c-f3a6ffe6a836 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.799833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1201/2387] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:44 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:58:44.806274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e1639-b4c5-499d-813c-f3a6ffe6a836 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.816094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f5e40-e6c0-4dd1-8c8a-d2ab546464ef admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.865307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa711e-2871-4e2e-83a5-bcb96b7bca6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.865919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa711e-2871-4e2e-83a5-bcb96b7bca6d service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.866191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa711e-2871-4e2e-83a5-bcb96b7bca6d service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.866456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa711e-2871-4e2e-83a5-bcb96b7bca6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.869501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e1639-b4c5-499d-813c-f3a6ffe6a836 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.874073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a9c1d-b31e-4b61-9e20-4b1445fc3789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.874670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a9c1d-b31e-4b61-9e20-4b1445fc3789 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.874949 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a9c1d-b31e-4b61-9e20-4b1445fc3789 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.875223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a9c1d-b31e-4b61-9e20-4b1445fc3789 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.875673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa711e-2871-4e2e-83a5-bcb96b7bca6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.875976 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a9c1d-b31e-4b61-9e20-4b1445fc3789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.877122 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa711e-2871-4e2e-83a5-bcb96b7bca6d service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.877894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a9c1d-b31e-4b61-9e20-4b1445fc3789 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.887070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1187/2388] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:58:44.906972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1202/2389] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:44 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:44.908224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa711e-2871-4e2e-83a5-bcb96b7bca6d service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.919356 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa711e-2871-4e2e-83a5-bcb96b7bca6d service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.924097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae80d38-92a3-4735-8f26-4b17aefdbef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.924699 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80d38-92a3-4735-8f26-4b17aefdbef9 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.924975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80d38-92a3-4735-8f26-4b17aefdbef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.925244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80d38-92a3-4735-8f26-4b17aefdbef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.925881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae80d38-92a3-4735-8f26-4b17aefdbef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.928277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae80d38-92a3-4735-8f26-4b17aefdbef9 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.938980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1188/2390] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:44.944220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a9c1d-b31e-4b61-9e20-4b1445fc3789 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.955553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a9c1d-b31e-4b61-9e20-4b1445fc3789 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:44.961184 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e7ce8-501e-4475-8ebd-68e8fcf7c708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:44.961939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e7ce8-501e-4475-8ebd-68e8fcf7c708 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:44.962419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e7ce8-501e-4475-8ebd-68e8fcf7c708 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:44.962553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e7ce8-501e-4475-8ebd-68e8fcf7c708 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:44.963126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e7ce8-501e-4475-8ebd-68e8fcf7c708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:44.964834 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e7ce8-501e-4475-8ebd-68e8fcf7c708 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:44.977014 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1203/2391] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:44.981629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae80d38-92a3-4735-8f26-4b17aefdbef9 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:44.994579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae80d38-92a3-4735-8f26-4b17aefdbef9 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:45.000441 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354d287-b04a-4987-9b46-fe3c6617d8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:45.001124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d287-b04a-4987-9b46-fe3c6617d8eb admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.001455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d287-b04a-4987-9b46-fe3c6617d8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.001745 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d287-b04a-4987-9b46-fe3c6617d8eb admin admin] PATH_INFO: `/v3/projects/3a53e1c3be7142f0a1d4604207394f19/users/b9617217fafa47da861a0eb9c8ca8cc7/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:45.029626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354d287-b04a-4987-9b46-fe3c6617d8eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a53e1c3be7142f0a1d4604207394f19, user_id=b9617217fafa47da861a0eb9c8ca8cc7, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:45.032553 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354d287-b04a-4987-9b46-fe3c6617d8eb admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:45.052880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1189/2392] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:44 2022] PUT /identity/v3/projects/3a53e1c3be7142f0a1d4604207394f19/users/b9617217fafa47da861a0eb9c8ca8cc7/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:45.066094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7b558-9061-4da5-a07b-a0ae89e1e613 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.066385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7b558-9061-4da5-a07b-a0ae89e1e613 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.066663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7b558-9061-4da5-a07b-a0ae89e1e613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:45.104761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-58f7b558-9061-4da5-a07b-a0ae89e1e613 None None] MFA Rules not processed for user `b9617217fafa47da861a0eb9c8ca8cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:45.106125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-58f7b558-9061-4da5-a07b-a0ae89e1e613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:45.203365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1204/2393] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:45.209852 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354d287-b04a-4987-9b46-fe3c6617d8eb admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:45.281063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:45.287660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354d287-b04a-4987-9b46-fe3c6617d8eb admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:45.313423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb148424-c5b7-4f9f-92fe-3ee5889b274d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:45.313980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb148424-c5b7-4f9f-92fe-3ee5889b274d admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.314262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb148424-c5b7-4f9f-92fe-3ee5889b274d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.314536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb148424-c5b7-4f9f-92fe-3ee5889b274d admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:45.315176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb148424-c5b7-4f9f-92fe-3ee5889b274d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:45.318123 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb148424-c5b7-4f9f-92fe-3ee5889b274d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:45.353818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1190/2394] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:45 2022] POST /identity/v3/projects => generated 405 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:58:45.357932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7b558-9061-4da5-a07b-a0ae89e1e613 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:45.370328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7b558-9061-4da5-a07b-a0ae89e1e613 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:45.431383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:45.457407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814def8-2ba8-4050-91c3-fe6c45956131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:45.457407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814def8-2ba8-4050-91c3-fe6c45956131 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.457407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814def8-2ba8-4050-91c3-fe6c45956131 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.457407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814def8-2ba8-4050-91c3-fe6c45956131 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:45.458387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814def8-2ba8-4050-91c3-fe6c45956131 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:45.458696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814def8-2ba8-4050-91c3-fe6c45956131 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:45.485633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1205/2395] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:45 2022] POST /identity/v3/users => generated 309 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:45.490174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb148424-c5b7-4f9f-92fe-3ee5889b274d admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:45.502091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb148424-c5b7-4f9f-92fe-3ee5889b274d admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:45.509935 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb1327-2efe-49e8-bc33-2805e786a661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:45.510816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb1327-2efe-49e8-bc33-2805e786a661 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.511381 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb1327-2efe-49e8-bc33-2805e786a661 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.511640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb1327-2efe-49e8-bc33-2805e786a661 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:45.512656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb1327-2efe-49e8-bc33-2805e786a661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:45.515193 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb1327-2efe-49e8-bc33-2805e786a661 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:45.526270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1191/2396] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:45.530679 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814def8-2ba8-4050-91c3-fe6c45956131 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:45.541901 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814def8-2ba8-4050-91c3-fe6c45956131 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:45.546618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c13ac-13a0-4209-abd4-04b745528e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:45.547519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c13ac-13a0-4209-abd4-04b745528e26 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.547999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c13ac-13a0-4209-abd4-04b745528e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.548275 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c13ac-13a0-4209-abd4-04b745528e26 admin admin] PATH_INFO: `/v3/projects/873da1857f9a45349dfdb367d60cbdae/users/f2a4149214f248cb9febd6ef7459ced9/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:45.566265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c13ac-13a0-4209-abd4-04b745528e26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=873da1857f9a45349dfdb367d60cbdae, user_id=f2a4149214f248cb9febd6ef7459ced9, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:45.569006 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c13ac-13a0-4209-abd4-04b745528e26 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:45.583894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1206/2397] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:45 2022] PUT /identity/v3/projects/873da1857f9a45349dfdb367d60cbdae/users/f2a4149214f248cb9febd6ef7459ced9/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:45.587264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb1327-2efe-49e8-bc33-2805e786a661 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:45.597658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb1327-2efe-49e8-bc33-2805e786a661 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:45.649366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762ea9b-f97e-4997-9061-1f86b40d351d None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.649890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762ea9b-f97e-4997-9061-1f86b40d351d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.650334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762ea9b-f97e-4997-9061-1f86b40d351d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:45.673288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d2e4a-1944-45db-861c-585175f371ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:45.675034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d2e4a-1944-45db-861c-585175f371ee admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.675518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d2e4a-1944-45db-861c-585175f371ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.675990 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d2e4a-1944-45db-861c-585175f371ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:45.677202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d2e4a-1944-45db-861c-585175f371ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:45.678848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d2e4a-1944-45db-861c-585175f371ee admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:45.682735 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4762ea9b-f97e-4997-9061-1f86b40d351d None None] MFA Rules not processed for user `a29226e406e844068e7c2f123d0aba7b`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:45.684019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4762ea9b-f97e-4997-9061-1f86b40d351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:45.693160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1192/2398] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:45.697647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d2e4a-1944-45db-861c-585175f371ee admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:45.709582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d2e4a-1944-45db-861c-585175f371ee admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:45.714578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8542f5-453f-45f4-b9bf-4f75661d03d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:45.715247 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8542f5-453f-45f4-b9bf-4f75661d03d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.715664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8542f5-453f-45f4-b9bf-4f75661d03d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.715934 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8542f5-453f-45f4-b9bf-4f75661d03d2 admin admin] PATH_INFO: `/v3/projects/873da1857f9a45349dfdb367d60cbdae/users/f2a4149214f248cb9febd6ef7459ced9/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:45.717843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8542f5-453f-45f4-b9bf-4f75661d03d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=873da1857f9a45349dfdb367d60cbdae, user_id=f2a4149214f248cb9febd6ef7459ced9, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:45.720534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8542f5-453f-45f4-b9bf-4f75661d03d2 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:45.733039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1193/2399] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:45 2022] PUT /identity/v3/projects/873da1857f9a45349dfdb367d60cbdae/users/f2a4149214f248cb9febd6ef7459ced9/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:45.744158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a007e-6d87-42b7-8709-8985b09cbee7 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.745132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a007e-6d87-42b7-8709-8985b09cbee7 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.745687 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a007e-6d87-42b7-8709-8985b09cbee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:45.771548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1207/2400] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:45.776155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762ea9b-f97e-4997-9061-1f86b40d351d None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:45.777809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-856a007e-6d87-42b7-8709-8985b09cbee7 None None] MFA Rules not processed for user `f2a4149214f248cb9febd6ef7459ced9`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:45.779088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-856a007e-6d87-42b7-8709-8985b09cbee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:45.788582 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762ea9b-f97e-4997-9061-1f86b40d351d None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:45.860786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1194/2401] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:45.861866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f776c1-9114-4f19-b937-9668f2ccc301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:45.862476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f776c1-9114-4f19-b937-9668f2ccc301 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.862752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f776c1-9114-4f19-b937-9668f2ccc301 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.863020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f776c1-9114-4f19-b937-9668f2ccc301 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:45.864410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f776c1-9114-4f19-b937-9668f2ccc301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:45.866313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f776c1-9114-4f19-b937-9668f2ccc301 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:45.872013 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a007e-6d87-42b7-8709-8985b09cbee7 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:45.873496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-856a007e-6d87-42b7-8709-8985b09cbee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:45.909377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1208/2402] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:45 2022] POST /identity/v3/projects => generated 405 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:45.910435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f776c1-9114-4f19-b937-9668f2ccc301 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:45.921874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:45.934598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:45.939227 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a007e-6d87-42b7-8709-8985b09cbee7 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:45.978645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f776c1-9114-4f19-b937-9668f2ccc301 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:45.983916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515345d5-e86f-46c2-b6a9-f240328f0ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:45.984609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515345d5-e86f-46c2-b6a9-f240328f0ecb admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.985169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515345d5-e86f-46c2-b6a9-f240328f0ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.985441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515345d5-e86f-46c2-b6a9-f240328f0ecb admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:45.986197 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515345d5-e86f-46c2-b6a9-f240328f0ecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:45.987790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca18ae-d112-4606-9e3f-1431bb0333a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:45.988198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515345d5-e86f-46c2-b6a9-f240328f0ecb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:45.988402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca18ae-d112-4606-9e3f-1431bb0333a0 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:45.988683 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca18ae-d112-4606-9e3f-1431bb0333a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:45.989586 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca18ae-d112-4606-9e3f-1431bb0333a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:46.013781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1209/2403] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:45 2022] POST /identity/v3/users => generated 309 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:46.017693 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515345d5-e86f-46c2-b6a9-f240328f0ecb admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:46.027463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515345d5-e86f-46c2-b6a9-f240328f0ecb admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:46.033786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7730d74-d363-4b81-ba13-28ad6d6169e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:46.034548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7730d74-d363-4b81-ba13-28ad6d6169e3 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:46.034955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7730d74-d363-4b81-ba13-28ad6d6169e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:46.035231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7730d74-d363-4b81-ba13-28ad6d6169e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:46.036035 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7730d74-d363-4b81-ba13-28ad6d6169e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:46.037891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7730d74-d363-4b81-ba13-28ad6d6169e3 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:46.052641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca18ae-d112-4606-9e3f-1431bb0333a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:46.052806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1210/2404] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:58:46.054172 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca18ae-d112-4606-9e3f-1431bb0333a0 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:46.056830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7730d74-d363-4b81-ba13-28ad6d6169e3 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:46.064967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1195/2405] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:45 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:58:46.068094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7730d74-d363-4b81-ba13-28ad6d6169e3 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:46.072903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa219f-5819-41fa-828c-c5e0bf707766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:46.073325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa219f-5819-41fa-828c-c5e0bf707766 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:46.073604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa219f-5819-41fa-828c-c5e0bf707766 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:46.073905 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa219f-5819-41fa-828c-c5e0bf707766 admin admin] PATH_INFO: `/v3/projects/b83bf669f3f14780b16d6ec17cfa89bc/users/c84fa0be95ba446f814d15a112f56ffe/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:46.092929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa219f-5819-41fa-828c-c5e0bf707766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83bf669f3f14780b16d6ec17cfa89bc, user_id=c84fa0be95ba446f814d15a112f56ffe, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:46.096102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa219f-5819-41fa-828c-c5e0bf707766 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:46.111241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1211/2406] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:46 2022] PUT /identity/v3/projects/b83bf669f3f14780b16d6ec17cfa89bc/users/c84fa0be95ba446f814d15a112f56ffe/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:46.115196 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca18ae-d112-4606-9e3f-1431bb0333a0 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:46.127573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca18ae-d112-4606-9e3f-1431bb0333a0 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:46.194129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e53883-0936-443a-87f2-84f1f677d6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:46.194762 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e53883-0936-443a-87f2-84f1f677d6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:46.195062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e53883-0936-443a-87f2-84f1f677d6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:46.195305 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e53883-0936-443a-87f2-84f1f677d6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:46.195996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e53883-0936-443a-87f2-84f1f677d6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:46.197601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e53883-0936-443a-87f2-84f1f677d6d3 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:46.207303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1196/2407] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:46.212150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa219f-5819-41fa-828c-c5e0bf707766 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:46.217224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e53883-0936-443a-87f2-84f1f677d6d3 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:46.222593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa219f-5819-41fa-828c-c5e0bf707766 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:46.228015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db37863-4360-4d57-ac25-dfed11609dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:46.228519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e53883-0936-443a-87f2-84f1f677d6d3 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:46.228988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db37863-4360-4d57-ac25-dfed11609dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:46.229338 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db37863-4360-4d57-ac25-dfed11609dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:46.229679 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db37863-4360-4d57-ac25-dfed11609dd5 admin admin] PATH_INFO: `/v3/projects/b83bf669f3f14780b16d6ec17cfa89bc/users/c84fa0be95ba446f814d15a112f56ffe/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:46.231562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db37863-4360-4d57-ac25-dfed11609dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83bf669f3f14780b16d6ec17cfa89bc, user_id=c84fa0be95ba446f814d15a112f56ffe, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:46.235043 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:46.235661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:46.235974 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:46.236245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] PATH_INFO: `/v3/users/b9617217fafa47da861a0eb9c8ca8cc7` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:46.237377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9617217fafa47da861a0eb9c8ca8cc7)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:46.238346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db37863-4360-4d57-ac25-dfed11609dd5 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:46.239346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:46.249906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1212/2408] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:46 2022] PUT /identity/v3/projects/b83bf669f3f14780b16d6ec17cfa89bc/users/c84fa0be95ba446f814d15a112f56ffe/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:46.278793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:46.289414 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9617217fafa47da861a0eb9c8ca8cc7'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:46.296524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9617217fafa47da861a0eb9c8ca8cc7'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:46.297239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9617217fafa47da861a0eb9c8ca8cc7'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:46.297873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9617217fafa47da861a0eb9c8ca8cc7'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:46.310226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9617217fafa47da861a0eb9c8ca8cc7'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:46.327424 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1197/2409] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:46 2022] DELETE /identity/v3/users/b9617217fafa47da861a0eb9c8ca8cc7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:46.331962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f6fbc-a2fa-4237-baaf-15709d1eaa0b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:46.332244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f6fbc-a2fa-4237-baaf-15709d1eaa0b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:46.332609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f6fbc-a2fa-4237-baaf-15709d1eaa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:46.354687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-f84f6fbc-a2fa-4237-baaf-15709d1eaa0b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:46.355997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f84f6fbc-a2fa-4237-baaf-15709d1eaa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:46.433736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1213/2410] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:46.445668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:46.447716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:46.518877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0b728-95f0-4598-a2da-cee7eda1cb9e admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:46.523886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc036d-fe8b-4380-b22f-10e6affa2ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:46.524769 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc036d-fe8b-4380-b22f-10e6affa2ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:46.525111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc036d-fe8b-4380-b22f-10e6affa2ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:46.525433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc036d-fe8b-4380-b22f-10e6affa2ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:46.537298 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc036d-fe8b-4380-b22f-10e6affa2ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:46.538792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc036d-fe8b-4380-b22f-10e6affa2ef5 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:46.550256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1198/2411] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:47.019879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f6fbc-a2fa-4237-baaf-15709d1eaa0b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:47.021932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f84f6fbc-a2fa-4237-baaf-15709d1eaa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:47.034030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f6fbc-a2fa-4237-baaf-15709d1eaa0b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:47.039542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:47.040330 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:47.040668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:47.040900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] PATH_INFO: `/v3/projects/3a53e1c3be7142f0a1d4604207394f19` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:47.042559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a53e1c3be7142f0a1d4604207394f19)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:47.044613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:47.117674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a53e1c3be7142f0a1d4604207394f19', 'request_id': 'req-3428e7f2-487b-4591-a444-1ed097036eeb'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.119218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75822854-834f-4497-b275-d60f816fd9ec None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:47.119706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75822854-834f-4497-b275-d60f816fd9ec None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:47.119817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75822854-834f-4497-b275-d60f816fd9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:47.124068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:47.125839 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a53e1c3be7142f0a1d4604207394f19', 'request_id': 'req-3428e7f2-487b-4591-a444-1ed097036eeb'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.142479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a53e1c3be7142f0a1d4604207394f19', 'request_id': 'req-3428e7f2-487b-4591-a444-1ed097036eeb'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.151381 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-75822854-834f-4497-b275-d60f816fd9ec None None] MFA Rules not processed for user `c84fa0be95ba446f814d15a112f56ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:47.152659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-75822854-834f-4497-b275-d60f816fd9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:47.157667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] The token cache is being invalidate because project 3a53e1c3be7142f0a1d4604207394f19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:47.158327 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.158571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.163581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1214/2412] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:47 2022] DELETE /identity/v3/projects/3a53e1c3be7142f0a1d4604207394f19 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:47.165479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:47.167204 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:47.233195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1199/2413] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:47.258398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428e7f2-487b-4591-a444-1ed097036eeb admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:47.263880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05dbb3f-7ce0-4353-bb20-cea1063ca10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:47.264646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dbb3f-7ce0-4353-bb20-cea1063ca10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:47.264921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dbb3f-7ce0-4353-bb20-cea1063ca10c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:47.265191 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dbb3f-7ce0-4353-bb20-cea1063ca10c admin admin] PATH_INFO: `/v3/users/f2a4149214f248cb9febd6ef7459ced9` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:47.267024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05dbb3f-7ce0-4353-bb20-cea1063ca10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2a4149214f248cb9febd6ef7459ced9)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:47.268887 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05dbb3f-7ce0-4353-bb20-cea1063ca10c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:47.327359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-f05dbb3f-7ce0-4353-bb20-cea1063ca10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2a4149214f248cb9febd6ef7459ced9'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.335761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-f05dbb3f-7ce0-4353-bb20-cea1063ca10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2a4149214f248cb9febd6ef7459ced9'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.340205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-f05dbb3f-7ce0-4353-bb20-cea1063ca10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2a4149214f248cb9febd6ef7459ced9'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.354902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-f05dbb3f-7ce0-4353-bb20-cea1063ca10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2a4149214f248cb9febd6ef7459ced9'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.364632 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-f05dbb3f-7ce0-4353-bb20-cea1063ca10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2a4149214f248cb9febd6ef7459ced9'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.371167 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1215/2414] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:47 2022] DELETE /identity/v3/users/f2a4149214f248cb9febd6ef7459ced9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:47.672537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75822854-834f-4497-b275-d60f816fd9ec None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:47.689160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75822854-834f-4497-b275-d60f816fd9ec None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:47.746482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:47.786185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe994ebf-fe2a-42e5-b457-3d90917f840d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:47.787358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe994ebf-fe2a-42e5-b457-3d90917f840d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:47.787629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe994ebf-fe2a-42e5-b457-3d90917f840d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:47.788056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe994ebf-fe2a-42e5-b457-3d90917f840d admin admin] PATH_INFO: `/v3/projects/873da1857f9a45349dfdb367d60cbdae` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:47.789201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe994ebf-fe2a-42e5-b457-3d90917f840d admin admin] RBAC: Authorizing `identity:delete_project(project_id=873da1857f9a45349dfdb367d60cbdae)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:47.792691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe994ebf-fe2a-42e5-b457-3d90917f840d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:47.860544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-fe994ebf-fe2a-42e5-b457-3d90917f840d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '873da1857f9a45349dfdb367d60cbdae', 'request_id': 'req-fe994ebf-fe2a-42e5-b457-3d90917f840d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.869065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-fe994ebf-fe2a-42e5-b457-3d90917f840d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '873da1857f9a45349dfdb367d60cbdae', 'request_id': 'req-fe994ebf-fe2a-42e5-b457-3d90917f840d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.878949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-fe994ebf-fe2a-42e5-b457-3d90917f840d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '873da1857f9a45349dfdb367d60cbdae', 'request_id': 'req-fe994ebf-fe2a-42e5-b457-3d90917f840d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.931168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-fe994ebf-fe2a-42e5-b457-3d90917f840d admin admin] The token cache is being invalidate because project 873da1857f9a45349dfdb367d60cbdae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:47.931168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-fe994ebf-fe2a-42e5-b457-3d90917f840d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.931168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-fe994ebf-fe2a-42e5-b457-3d90917f840d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:47.931168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1200/2415] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:47 2022] DELETE /identity/v3/projects/873da1857f9a45349dfdb367d60cbdae => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:47.932634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13593aac-5a9a-465d-8599-d3c52772aa07 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:47.932634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13593aac-5a9a-465d-8599-d3c52772aa07 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:47.932634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13593aac-5a9a-465d-8599-d3c52772aa07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:47.952064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-13593aac-5a9a-465d-8599-d3c52772aa07 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:47.954393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-13593aac-5a9a-465d-8599-d3c52772aa07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:48.034268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1216/2416] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:48.038618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c36384-85f8-4013-8de6-55e4e2bae90a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.038909 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c36384-85f8-4013-8de6-55e4e2bae90a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.039189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c36384-85f8-4013-8de6-55e4e2bae90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.061292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-24c36384-85f8-4013-8de6-55e4e2bae90a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:48.062570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-24c36384-85f8-4013-8de6-55e4e2bae90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:48.071568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1201/2417] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:48.076017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13593aac-5a9a-465d-8599-d3c52772aa07 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.082803 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:48.087570 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13593aac-5a9a-465d-8599-d3c52772aa07 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.093643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00c6f4-bf64-490d-81ac-abbaa053270f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.094285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00c6f4-bf64-490d-81ac-abbaa053270f admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.094734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00c6f4-bf64-490d-81ac-abbaa053270f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.095176 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00c6f4-bf64-490d-81ac-abbaa053270f admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.096153 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00c6f4-bf64-490d-81ac-abbaa053270f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:58:48.096458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00c6f4-bf64-490d-81ac-abbaa053270f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.098102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00c6f4-bf64-490d-81ac-abbaa053270f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.112774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1217/2418] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:58:48.123126 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c36384-85f8-4013-8de6-55e4e2bae90a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.133651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c36384-85f8-4013-8de6-55e4e2bae90a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.140202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d808b9-c076-4936-bdae-cdce4f99e774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.140413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d808b9-c076-4936-bdae-cdce4f99e774 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.140718 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d808b9-c076-4936-bdae-cdce4f99e774 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.141030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d808b9-c076-4936-bdae-cdce4f99e774 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.142017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d808b9-c076-4936-bdae-cdce4f99e774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.143615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d808b9-c076-4936-bdae-cdce4f99e774 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.171553 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1202/2419] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:48 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:58:48.175830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00c6f4-bf64-490d-81ac-abbaa053270f admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.187375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00c6f4-bf64-490d-81ac-abbaa053270f admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.249262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50530ed-f65c-4555-b1d2-a3347122c4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.250003 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50530ed-f65c-4555-b1d2-a3347122c4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.250292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50530ed-f65c-4555-b1d2-a3347122c4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.250687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50530ed-f65c-4555-b1d2-a3347122c4d1 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.251334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50530ed-f65c-4555-b1d2-a3347122c4d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.253317 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50530ed-f65c-4555-b1d2-a3347122c4d1 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.277325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1218/2420] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:48 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:48.281560 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d808b9-c076-4936-bdae-cdce4f99e774 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.296129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d808b9-c076-4936-bdae-cdce4f99e774 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.301417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502edcb-d85d-4c67-9828-bc42f72c6b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.302203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502edcb-d85d-4c67-9828-bc42f72c6b96 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.302427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502edcb-d85d-4c67-9828-bc42f72c6b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.302818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502edcb-d85d-4c67-9828-bc42f72c6b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.303730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502edcb-d85d-4c67-9828-bc42f72c6b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.305924 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502edcb-d85d-4c67-9828-bc42f72c6b96 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.317515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1203/2421] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:48.321350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50530ed-f65c-4555-b1d2-a3347122c4d1 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.331462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50530ed-f65c-4555-b1d2-a3347122c4d1 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.337105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56181c-6198-4762-8e25-b59bce79867f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.337712 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56181c-6198-4762-8e25-b59bce79867f admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.337990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56181c-6198-4762-8e25-b59bce79867f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.338261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56181c-6198-4762-8e25-b59bce79867f admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.338902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56181c-6198-4762-8e25-b59bce79867f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.340536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56181c-6198-4762-8e25-b59bce79867f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.352280 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1219/2422] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:48.356353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502edcb-d85d-4c67-9828-bc42f72c6b96 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.366842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502edcb-d85d-4c67-9828-bc42f72c6b96 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.372766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4190d7-9509-45f5-9eb4-4958f980bfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.373480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4190d7-9509-45f5-9eb4-4958f980bfa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.373760 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4190d7-9509-45f5-9eb4-4958f980bfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.374037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4190d7-9509-45f5-9eb4-4958f980bfa6 admin admin] PATH_INFO: `/v3/projects/631c8fcc50a74d14a62749039515b5a9/users/10a86d22ad7f4362a6b846b296639331/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.392096 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4190d7-9509-45f5-9eb4-4958f980bfa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=631c8fcc50a74d14a62749039515b5a9, user_id=10a86d22ad7f4362a6b846b296639331, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.394805 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4190d7-9509-45f5-9eb4-4958f980bfa6 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.408451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1204/2423] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:48 2022] PUT /identity/v3/projects/631c8fcc50a74d14a62749039515b5a9/users/10a86d22ad7f4362a6b846b296639331/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:48.420284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea15300-399b-458e-bc5f-e815f21f9ae0 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.420577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea15300-399b-458e-bc5f-e815f21f9ae0 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.420855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea15300-399b-458e-bc5f-e815f21f9ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.450031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4190d7-9509-45f5-9eb4-4958f980bfa6 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.453438 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-fb4190d7-9509-45f5-9eb4-4958f980bfa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:48.456096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-dea15300-399b-458e-bc5f-e815f21f9ae0 None None] MFA Rules not processed for user `10a86d22ad7f4362a6b846b296639331`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:48.457287 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-dea15300-399b-458e-bc5f-e815f21f9ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:48.519730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4190d7-9509-45f5-9eb4-4958f980bfa6 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.524999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.525644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.525863 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.526141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] PATH_INFO: `/v3/users/a29226e406e844068e7c2f123d0aba7b` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.526370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1220/2424] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:48.527311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a29226e406e844068e7c2f123d0aba7b)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.529261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.530567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea15300-399b-458e-bc5f-e815f21f9ae0 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.534863 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:48.540840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea15300-399b-458e-bc5f-e815f21f9ae0 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.545555 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542efd3-c8c3-42e6-83ed-c7ca2d5160f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.546203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542efd3-c8c3-42e6-83ed-c7ca2d5160f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.546482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542efd3-c8c3-42e6-83ed-c7ca2d5160f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.546752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542efd3-c8c3-42e6-83ed-c7ca2d5160f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.547364 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542efd3-c8c3-42e6-83ed-c7ca2d5160f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.549285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542efd3-c8c3-42e6-83ed-c7ca2d5160f9 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.577336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1221/2425] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:48 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:48.578274 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a29226e406e844068e7c2f123d0aba7b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:48.581208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542efd3-c8c3-42e6-83ed-c7ca2d5160f9 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.586813 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a29226e406e844068e7c2f123d0aba7b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:48.587511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a29226e406e844068e7c2f123d0aba7b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:48.588209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a29226e406e844068e7c2f123d0aba7b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:48.600093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a29226e406e844068e7c2f123d0aba7b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:48.613796 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1205/2426] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:48 2022] DELETE /identity/v3/users/a29226e406e844068e7c2f123d0aba7b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:48.624913 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:48.649113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542efd3-c8c3-42e6-83ed-c7ca2d5160f9 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.712487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b034417-6ddf-4a66-b0d3-375dae66b792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.713045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b034417-6ddf-4a66-b0d3-375dae66b792 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.713323 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b034417-6ddf-4a66-b0d3-375dae66b792 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.713601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b034417-6ddf-4a66-b0d3-375dae66b792 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.714224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b034417-6ddf-4a66-b0d3-375dae66b792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.716185 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b034417-6ddf-4a66-b0d3-375dae66b792 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.741504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1222/2427] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:48 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:48.745424 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.746817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:48.759455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980d053-5dfe-429e-a86b-a13d76e704e7 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.764319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8888d0a-e71b-4950-bb63-7295ed09b8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.764914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8888d0a-e71b-4950-bb63-7295ed09b8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.765189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8888d0a-e71b-4950-bb63-7295ed09b8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.765457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8888d0a-e71b-4950-bb63-7295ed09b8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.766085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8888d0a-e71b-4950-bb63-7295ed09b8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.767668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8888d0a-e71b-4950-bb63-7295ed09b8b4 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.778312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1206/2428] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:48.782287 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b034417-6ddf-4a66-b0d3-375dae66b792 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.796179 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8888d0a-e71b-4950-bb63-7295ed09b8b4 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.796902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b034417-6ddf-4a66-b0d3-375dae66b792 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.798039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f8888d0a-e71b-4950-bb63-7295ed09b8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:48.801958 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec52802-5096-40c0-9263-ee1ad00b8b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.802566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec52802-5096-40c0-9263-ee1ad00b8b11 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.807578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec52802-5096-40c0-9263-ee1ad00b8b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.807578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec52802-5096-40c0-9263-ee1ad00b8b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.807578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec52802-5096-40c0-9263-ee1ad00b8b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.807578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec52802-5096-40c0-9263-ee1ad00b8b11 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.815266 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8888d0a-e71b-4950-bb63-7295ed09b8b4 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.821091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1223/2429] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:58:48.821272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.821960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.822162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.822449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] PATH_INFO: `/v3/projects/929725e14c9e499ca35214ffd8b8a611` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.825279 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=929725e14c9e499ca35214ffd8b8a611)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.826026 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec52802-5096-40c0-9263-ee1ad00b8b11 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.827100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.836558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec52802-5096-40c0-9263-ee1ad00b8b11 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.841328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe772b-e46e-4bc5-809a-3a15210774e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:48.842002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe772b-e46e-4bc5-809a-3a15210774e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.842293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe772b-e46e-4bc5-809a-3a15210774e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.842563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe772b-e46e-4bc5-809a-3a15210774e4 admin admin] PATH_INFO: `/v3/projects/9d84a70e1ec045868bd4d47f9f9818da/users/b013d1979c594747895a91f63d0ad4a5/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.859830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe772b-e46e-4bc5-809a-3a15210774e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d84a70e1ec045868bd4d47f9f9818da, user_id=b013d1979c594747895a91f63d0ad4a5, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:48.862495 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe772b-e46e-4bc5-809a-3a15210774e4 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:48.876452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1224/2430] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:48 2022] PUT /identity/v3/projects/9d84a70e1ec045868bd4d47f9f9818da/users/b013d1979c594747895a91f63d0ad4a5/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:48.885324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '929725e14c9e499ca35214ffd8b8a611', 'request_id': 'req-b2471c76-c5bb-470a-bec5-cea4aef68d49'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:48.885324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '929725e14c9e499ca35214ffd8b8a611', 'request_id': 'req-b2471c76-c5bb-470a-bec5-cea4aef68d49'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:48.886687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377f042-e5af-4757-801b-5ad080e78af6 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:48.886969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377f042-e5af-4757-801b-5ad080e78af6 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:48.887233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377f042-e5af-4757-801b-5ad080e78af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:48.897559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '929725e14c9e499ca35214ffd8b8a611', 'request_id': 'req-b2471c76-c5bb-470a-bec5-cea4aef68d49'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:48.906673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] The token cache is being invalidate because project 929725e14c9e499ca35214ffd8b8a611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:48.906968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:48.907637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:48.909158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1207/2431] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:48 2022] DELETE /identity/v3/projects/929725e14c9e499ca35214ffd8b8a611 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:48.913594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:48.914911 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:48.919362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-d377f042-e5af-4757-801b-5ad080e78af6 None None] MFA Rules not processed for user `b013d1979c594747895a91f63d0ad4a5`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:48.920593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d377f042-e5af-4757-801b-5ad080e78af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:48.980642 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:48.984688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2471c76-c5bb-470a-bec5-cea4aef68d49 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:48.994453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1225/2432] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:48.999250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377f042-e5af-4757-801b-5ad080e78af6 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.000673 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d377f042-e5af-4757-801b-5ad080e78af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:49.033076 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:49.046845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.047477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.047802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.048105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] PATH_INFO: `/v3/users/c84fa0be95ba446f814d15a112f56ffe` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.049276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] RBAC: Authorizing `identity:delete_user(user_id=c84fa0be95ba446f814d15a112f56ffe)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.051772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.069507 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377f042-e5af-4757-801b-5ad080e78af6 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.086066 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef182b2-56a6-423f-9d9a-966a6174be0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.086798 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef182b2-56a6-423f-9d9a-966a6174be0b admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.087086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef182b2-56a6-423f-9d9a-966a6174be0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.087387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef182b2-56a6-423f-9d9a-966a6174be0b admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.088042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef182b2-56a6-423f-9d9a-966a6174be0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.090219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef182b2-56a6-423f-9d9a-966a6174be0b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.117032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c84fa0be95ba446f814d15a112f56ffe'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:49.120751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1226/2433] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:48 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:49.125125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef182b2-56a6-423f-9d9a-966a6174be0b admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.125450 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c84fa0be95ba446f814d15a112f56ffe'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:49.126231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c84fa0be95ba446f814d15a112f56ffe'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:49.126884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c84fa0be95ba446f814d15a112f56ffe'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:49.142391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c84fa0be95ba446f814d15a112f56ffe'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:49.156810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1208/2434] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:48 2022] DELETE /identity/v3/users/c84fa0be95ba446f814d15a112f56ffe => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:49.198998 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef182b2-56a6-423f-9d9a-966a6174be0b admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.279130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1925e5-6d5f-4155-b7bb-93e5d0cfd907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.279844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1925e5-6d5f-4155-b7bb-93e5d0cfd907 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.280127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1925e5-6d5f-4155-b7bb-93e5d0cfd907 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.280401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1925e5-6d5f-4155-b7bb-93e5d0cfd907 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.281026 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1925e5-6d5f-4155-b7bb-93e5d0cfd907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.282962 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1925e5-6d5f-4155-b7bb-93e5d0cfd907 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.306758 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1227/2435] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:49 2022] POST /identity/v3/users => generated 296 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:49.323469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.325222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:49.356390 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae206ed5-f6e7-4d02-8c28-e334801fbace admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.364290 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d0722-2d69-4ec8-b6e5-0a17f75159d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.364994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d0722-2d69-4ec8-b6e5-0a17f75159d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.365353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d0722-2d69-4ec8-b6e5-0a17f75159d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.365958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d0722-2d69-4ec8-b6e5-0a17f75159d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.367218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d0722-2d69-4ec8-b6e5-0a17f75159d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.367831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1925e5-6d5f-4155-b7bb-93e5d0cfd907 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.368999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d0722-2d69-4ec8-b6e5-0a17f75159d6 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.370230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8b1925e5-6d5f-4155-b7bb-93e5d0cfd907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:49.384304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1209/2436] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:49.389334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d0722-2d69-4ec8-b6e5-0a17f75159d6 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.401891 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1925e5-6d5f-4155-b7bb-93e5d0cfd907 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.406908 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d0722-2d69-4ec8-b6e5-0a17f75159d6 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.412240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f92e8-1bb6-439a-9b97-88661cf53343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.412511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f92e8-1bb6-439a-9b97-88661cf53343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.412931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f92e8-1bb6-439a-9b97-88661cf53343 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.413089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde36fe-eef0-41ed-9bab-7ea6d9da7351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.413243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f92e8-1bb6-439a-9b97-88661cf53343 admin admin] PATH_INFO: `/v3/projects/b83bf669f3f14780b16d6ec17cfa89bc` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.413639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde36fe-eef0-41ed-9bab-7ea6d9da7351 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.413984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde36fe-eef0-41ed-9bab-7ea6d9da7351 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.414421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde36fe-eef0-41ed-9bab-7ea6d9da7351 admin admin] PATH_INFO: `/v3/projects/9a887dd93e464a0e8eec3c2e6b32a814/users/da86b79414f64161bc18d7edd47ca499/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.414647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f92e8-1bb6-439a-9b97-88661cf53343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b83bf669f3f14780b16d6ec17cfa89bc)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.417287 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f92e8-1bb6-439a-9b97-88661cf53343 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.434125 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde36fe-eef0-41ed-9bab-7ea6d9da7351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a887dd93e464a0e8eec3c2e6b32a814, user_id=da86b79414f64161bc18d7edd47ca499, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.437021 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde36fe-eef0-41ed-9bab-7ea6d9da7351 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.451623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1210/2437] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:49 2022] PUT /identity/v3/projects/9a887dd93e464a0e8eec3c2e6b32a814/users/da86b79414f64161bc18d7edd47ca499/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:49.455493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde36fe-eef0-41ed-9bab-7ea6d9da7351 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.469664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:49.474072 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-400f92e8-1bb6-439a-9b97-88661cf53343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b83bf669f3f14780b16d6ec17cfa89bc', 'request_id': 'req-400f92e8-1bb6-439a-9b97-88661cf53343'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:49.481657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-400f92e8-1bb6-439a-9b97-88661cf53343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b83bf669f3f14780b16d6ec17cfa89bc', 'request_id': 'req-400f92e8-1bb6-439a-9b97-88661cf53343'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:49.490725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-400f92e8-1bb6-439a-9b97-88661cf53343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b83bf669f3f14780b16d6ec17cfa89bc', 'request_id': 'req-400f92e8-1bb6-439a-9b97-88661cf53343'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:49.502653 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-400f92e8-1bb6-439a-9b97-88661cf53343 admin admin] The token cache is being invalidate because project b83bf669f3f14780b16d6ec17cfa89bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:49.502871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-400f92e8-1bb6-439a-9b97-88661cf53343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:49.503563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-400f92e8-1bb6-439a-9b97-88661cf53343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:49.505144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1228/2438] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:49 2022] DELETE /identity/v3/projects/b83bf669f3f14780b16d6ec17cfa89bc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:49.509956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f14bd-4a39-4c11-9607-cbcbc42fddb1 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.510238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f14bd-4a39-4c11-9607-cbcbc42fddb1 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.510569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f14bd-4a39-4c11-9607-cbcbc42fddb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.512328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:49.529884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde36fe-eef0-41ed-9bab-7ea6d9da7351 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.532997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-527f14bd-4a39-4c11-9607-cbcbc42fddb1 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:49.534200 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-527f14bd-4a39-4c11-9607-cbcbc42fddb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:49.588002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67a974-e6a0-4d64-b047-b9b6d7eab43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.588611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67a974-e6a0-4d64-b047-b9b6d7eab43a admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.588896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67a974-e6a0-4d64-b047-b9b6d7eab43a admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.589286 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67a974-e6a0-4d64-b047-b9b6d7eab43a admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.589929 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67a974-e6a0-4d64-b047-b9b6d7eab43a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.591554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67a974-e6a0-4d64-b047-b9b6d7eab43a admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.594160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1229/2439] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:49.602076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1211/2440] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:58:49.602359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddd5eb-7c9f-450e-bf72-147406fa3331 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.602643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddd5eb-7c9f-450e-bf72-147406fa3331 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.602917 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddd5eb-7c9f-450e-bf72-147406fa3331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.605983 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67a974-e6a0-4d64-b047-b9b6d7eab43a admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.607671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-de67a974-e6a0-4d64-b047-b9b6d7eab43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:49.619906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67a974-e6a0-4d64-b047-b9b6d7eab43a admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.622018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-74ddd5eb-7c9f-450e-bf72-147406fa3331 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:49.623206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-74ddd5eb-7c9f-450e-bf72-147406fa3331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:49.624592 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c605534-e0a3-4f6a-9c53-0e5e619662ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.625230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c605534-e0a3-4f6a-9c53-0e5e619662ec admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.625537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c605534-e0a3-4f6a-9c53-0e5e619662ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.625810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c605534-e0a3-4f6a-9c53-0e5e619662ec admin admin] PATH_INFO: `/v3/projects/9a887dd93e464a0e8eec3c2e6b32a814/users/da86b79414f64161bc18d7edd47ca499/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.627737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c605534-e0a3-4f6a-9c53-0e5e619662ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a887dd93e464a0e8eec3c2e6b32a814, user_id=da86b79414f64161bc18d7edd47ca499, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.630401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c605534-e0a3-4f6a-9c53-0e5e619662ec admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.631130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1230/2441] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:49.635180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddd5eb-7c9f-450e-bf72-147406fa3331 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.641277 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1212/2442] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:49 2022] PUT /identity/v3/projects/9a887dd93e464a0e8eec3c2e6b32a814/users/da86b79414f64161bc18d7edd47ca499/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:49.646037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddd5eb-7c9f-450e-bf72-147406fa3331 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.654172 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8b2d-933d-4b7b-a3fe-02728c45b0bf None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.654465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8b2d-933d-4b7b-a3fe-02728c45b0bf None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.654750 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8b2d-933d-4b7b-a3fe-02728c45b0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.683039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-1f1b8b2d-933d-4b7b-a3fe-02728c45b0bf None None] MFA Rules not processed for user `da86b79414f64161bc18d7edd47ca499`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:49.684301 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1f1b8b2d-933d-4b7b-a3fe-02728c45b0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:49.702398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74649335-65f8-45ac-a0d9-00429820cd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.702997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74649335-65f8-45ac-a0d9-00429820cd0c admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.703273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74649335-65f8-45ac-a0d9-00429820cd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.703543 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74649335-65f8-45ac-a0d9-00429820cd0c admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.704219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74649335-65f8-45ac-a0d9-00429820cd0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:58:49.704755 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74649335-65f8-45ac-a0d9-00429820cd0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.706103 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74649335-65f8-45ac-a0d9-00429820cd0c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.718087 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1231/2443] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:58:49.722522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74649335-65f8-45ac-a0d9-00429820cd0c admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.734260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74649335-65f8-45ac-a0d9-00429820cd0c admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.739163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbc5cf-4e72-4fb4-9ec4-7d6a68fdb3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.739784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbc5cf-4e72-4fb4-9ec4-7d6a68fdb3e4 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.740063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbc5cf-4e72-4fb4-9ec4-7d6a68fdb3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.740335 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbc5cf-4e72-4fb4-9ec4-7d6a68fdb3e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.741022 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbc5cf-4e72-4fb4-9ec4-7d6a68fdb3e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.742920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbc5cf-4e72-4fb4-9ec4-7d6a68fdb3e4 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.755215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1213/2444] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:49.767915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b8b2d-933d-4b7b-a3fe-02728c45b0bf None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.769456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1f1b8b2d-933d-4b7b-a3fe-02728c45b0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:49.773260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1232/2445] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:49 2022] POST /identity/v3/projects => generated 357 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:49.777226 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbc5cf-4e72-4fb4-9ec4-7d6a68fdb3e4 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.822386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:49.828256 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b8b2d-933d-4b7b-a3fe-02728c45b0bf None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.844404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbc5cf-4e72-4fb4-9ec4-7d6a68fdb3e4 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.845472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:49.850597 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4fc5a2-bf0f-4b30-baa2-f89972201bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.851242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fc5a2-bf0f-4b30-baa2-f89972201bef admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.851514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fc5a2-bf0f-4b30-baa2-f89972201bef admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.851815 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fc5a2-bf0f-4b30-baa2-f89972201bef admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.852438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4fc5a2-bf0f-4b30-baa2-f89972201bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.854347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4fc5a2-bf0f-4b30-baa2-f89972201bef admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.878003 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1233/2446] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:49 2022] POST /identity/v3/users => generated 279 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:49.878996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4fc5a2-bf0f-4b30-baa2-f89972201bef admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.881117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4f789-a2f5-4a5d-8254-3f844369f33c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.881727 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4f789-a2f5-4a5d-8254-3f844369f33c service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.881999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4f789-a2f5-4a5d-8254-3f844369f33c service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.882266 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4f789-a2f5-4a5d-8254-3f844369f33c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.889432 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4fc5a2-bf0f-4b30-baa2-f89972201bef admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.894377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b6dc2-14c1-45ec-b834-29aa41760e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.895713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b6dc2-14c1-45ec-b834-29aa41760e98 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.896327 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b6dc2-14c1-45ec-b834-29aa41760e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.896607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b6dc2-14c1-45ec-b834-29aa41760e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.897256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b6dc2-14c1-45ec-b834-29aa41760e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.899611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b6dc2-14c1-45ec-b834-29aa41760e98 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.912032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1234/2447] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:58:49.915630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b6dc2-14c1-45ec-b834-29aa41760e98 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.925737 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b6dc2-14c1-45ec-b834-29aa41760e98 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.931210 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21de6b-4a0c-4359-871c-cbda569343e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.931850 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21de6b-4a0c-4359-871c-cbda569343e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.932257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21de6b-4a0c-4359-871c-cbda569343e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.932562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21de6b-4a0c-4359-871c-cbda569343e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.933471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21de6b-4a0c-4359-871c-cbda569343e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.934783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21de6b-4a0c-4359-871c-cbda569343e8 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.946047 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1235/2448] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:49.949494 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d4f789-a2f5-4a5d-8254-3f844369f33c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.950275 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21de6b-4a0c-4359-871c-cbda569343e8 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:49.951003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d4f789-a2f5-4a5d-8254-3f844369f33c service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:49.960050 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21de6b-4a0c-4359-871c-cbda569343e8 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:49.961297 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1214/2449] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:58:49.964856 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046e4de-e323-478b-83f5-52632d2249c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:49.965505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046e4de-e323-478b-83f5-52632d2249c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:49.965783 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046e4de-e323-478b-83f5-52632d2249c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:49.966052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046e4de-e323-478b-83f5-52632d2249c2 admin admin] PATH_INFO: `/v3/projects/ed8db0e68b354b739f071af29760345a/users/a3226c1153fb43ccbf3eb0a9749b7f7c/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:49.982561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046e4de-e323-478b-83f5-52632d2249c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed8db0e68b354b739f071af29760345a, user_id=a3226c1153fb43ccbf3eb0a9749b7f7c, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:49.985947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046e4de-e323-478b-83f5-52632d2249c2 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:50.004533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1236/2450] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:49 2022] PUT /identity/v3/projects/ed8db0e68b354b739f071af29760345a/users/a3226c1153fb43ccbf3eb0a9749b7f7c/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:50.005067 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4f789-a2f5-4a5d-8254-3f844369f33c service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:50.006590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-84d4f789-a2f5-4a5d-8254-3f844369f33c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:50.015052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13db0bd-e3ed-446b-9ab3-66b999b57ada None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:50.015336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13db0bd-e3ed-446b-9ab3-66b999b57ada None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:50.015604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13db0bd-e3ed-446b-9ab3-66b999b57ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:50.017772 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4f789-a2f5-4a5d-8254-3f844369f33c service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:50.044637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-f13db0bd-e3ed-446b-9ab3-66b999b57ada None None] MFA Rules not processed for user `a3226c1153fb43ccbf3eb0a9749b7f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:50.045963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f13db0bd-e3ed-446b-9ab3-66b999b57ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:50.067532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b314a-5fc0-4ad7-a4f0-37b7e3b75f42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:50.068170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b314a-5fc0-4ad7-a4f0-37b7e3b75f42 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:50.068470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b314a-5fc0-4ad7-a4f0-37b7e3b75f42 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:50.068736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b314a-5fc0-4ad7-a4f0-37b7e3b75f42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:50.070673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-967b314a-5fc0-4ad7-a4f0-37b7e3b75f42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:50.110462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1237/2451] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:50.115663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13db0bd-e3ed-446b-9ab3-66b999b57ada None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:50.131786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b314a-5fc0-4ad7-a4f0-37b7e3b75f42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:50.133319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b314a-5fc0-4ad7-a4f0-37b7e3b75f42 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:50.142631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1215/2452] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:58:50.186816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13db0bd-e3ed-446b-9ab3-66b999b57ada None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:50.192752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7549e-5f5c-49ef-b971-70459f961075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:50.193442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7549e-5f5c-49ef-b971-70459f961075 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:50.193757 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7549e-5f5c-49ef-b971-70459f961075 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:50.194008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7549e-5f5c-49ef-b971-70459f961075 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:50.194807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7549e-5f5c-49ef-b971-70459f961075 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:50.199105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7549e-5f5c-49ef-b971-70459f961075 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:50.221628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:50.232530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1238/2453] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:50 2022] POST /identity/v3/projects => generated 357 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:50.233807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b314a-5fc0-4ad7-a4f0-37b7e3b75f42 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:50.247459 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b314a-5fc0-4ad7-a4f0-37b7e3b75f42 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:50.268429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:50.304990 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5c7f7-5b12-499f-9381-0c5eef30ffdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:50.305594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5c7f7-5b12-499f-9381-0c5eef30ffdb admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:50.305871 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5c7f7-5b12-499f-9381-0c5eef30ffdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:50.306139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5c7f7-5b12-499f-9381-0c5eef30ffdb admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:50.306749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5c7f7-5b12-499f-9381-0c5eef30ffdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:50.308689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5c7f7-5b12-499f-9381-0c5eef30ffdb admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:50.329646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1216/2454] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:50 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:58:50.333868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7549e-5f5c-49ef-b971-70459f961075 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:50.346034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7549e-5f5c-49ef-b971-70459f961075 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:50.351124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e2c63-e976-4b23-9080-f8d4576374b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:50.351751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e2c63-e976-4b23-9080-f8d4576374b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:50.352030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e2c63-e976-4b23-9080-f8d4576374b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:50.352328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e2c63-e976-4b23-9080-f8d4576374b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:50.353001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e2c63-e976-4b23-9080-f8d4576374b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:50.354872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e2c63-e976-4b23-9080-f8d4576374b6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:50.366818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1239/2455] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:50.371239 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5c7f7-5b12-499f-9381-0c5eef30ffdb admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:50.382056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5c7f7-5b12-499f-9381-0c5eef30ffdb admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:50.386658 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4d260-397d-41e6-b327-87dfc27ace4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:50.387310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4d260-397d-41e6-b327-87dfc27ace4d admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:50.387573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4d260-397d-41e6-b327-87dfc27ace4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:50.387870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4d260-397d-41e6-b327-87dfc27ace4d admin admin] PATH_INFO: `/v3/projects/60f7c4818b4c41f6ab6777e560397a42/users/dbc7ae2daeea49e49115a587d59bbc30/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:50.404683 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4d260-397d-41e6-b327-87dfc27ace4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f7c4818b4c41f6ab6777e560397a42, user_id=dbc7ae2daeea49e49115a587d59bbc30, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:50.407343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4d260-397d-41e6-b327-87dfc27ace4d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:50.422499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1217/2456] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:50 2022] PUT /identity/v3/projects/60f7c4818b4c41f6ab6777e560397a42/users/dbc7ae2daeea49e49115a587d59bbc30/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:50.428571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e2c63-e976-4b23-9080-f8d4576374b6 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:50.442990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e2c63-e976-4b23-9080-f8d4576374b6 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:50.532751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a3101-cd6a-47e0-9079-b8e16f49724a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:50.533894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a3101-cd6a-47e0-9079-b8e16f49724a admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:50.534393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a3101-cd6a-47e0-9079-b8e16f49724a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:50.535162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a3101-cd6a-47e0-9079-b8e16f49724a admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:50.536093 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a3101-cd6a-47e0-9079-b8e16f49724a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:50.538107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a3101-cd6a-47e0-9079-b8e16f49724a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:50.549901 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1240/2457] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:50.554614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4d260-397d-41e6-b327-87dfc27ace4d admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:50.569691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4d260-397d-41e6-b327-87dfc27ace4d admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:50.574856 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d122d3-9bc2-4cc5-a563-aa79c4c170a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:50.575580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d122d3-9bc2-4cc5-a563-aa79c4c170a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:50.575897 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d122d3-9bc2-4cc5-a563-aa79c4c170a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:50.576138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d122d3-9bc2-4cc5-a563-aa79c4c170a8 admin admin] PATH_INFO: `/v3/projects/60f7c4818b4c41f6ab6777e560397a42/users/dbc7ae2daeea49e49115a587d59bbc30/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:50.578046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d122d3-9bc2-4cc5-a563-aa79c4c170a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f7c4818b4c41f6ab6777e560397a42, user_id=dbc7ae2daeea49e49115a587d59bbc30, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:50.580752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d122d3-9bc2-4cc5-a563-aa79c4c170a8 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:50.591481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1218/2458] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:50 2022] PUT /identity/v3/projects/60f7c4818b4c41f6ab6777e560397a42/users/dbc7ae2daeea49e49115a587d59bbc30/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:50.604995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d57520-b3a6-4400-a2b8-92b38f39ded2 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:50.605281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d57520-b3a6-4400-a2b8-92b38f39ded2 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:50.605554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d57520-b3a6-4400-a2b8-92b38f39ded2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:50.634482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-01d57520-b3a6-4400-a2b8-92b38f39ded2 None None] MFA Rules not processed for user `dbc7ae2daeea49e49115a587d59bbc30`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:50.635713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-01d57520-b3a6-4400-a2b8-92b38f39ded2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:50.708715 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1241/2459] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:50 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:50.720620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d122d3-9bc2-4cc5-a563-aa79c4c170a8 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:50.783281 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d122d3-9bc2-4cc5-a563-aa79c4c170a8 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:50.788777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c10232-8f46-4368-b9dc-8aecb1a424e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:50.789600 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c10232-8f46-4368-b9dc-8aecb1a424e7 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:50.789885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c10232-8f46-4368-b9dc-8aecb1a424e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:50.790235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c10232-8f46-4368-b9dc-8aecb1a424e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:50.802803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c10232-8f46-4368-b9dc-8aecb1a424e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:50.804404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c10232-8f46-4368-b9dc-8aecb1a424e7 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:50.813922 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1219/2460] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:50 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:50.897125 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d57520-b3a6-4400-a2b8-92b38f39ded2 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:50.908002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d57520-b3a6-4400-a2b8-92b38f39ded2 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:50.912251 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced3f01-9625-4116-992d-d3a5861d145a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:50.912852 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced3f01-9625-4116-992d-d3a5861d145a service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:50.913127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced3f01-9625-4116-992d-d3a5861d145a service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:50.913400 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced3f01-9625-4116-992d-d3a5861d145a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:50.989883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced3f01-9625-4116-992d-d3a5861d145a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:50.991372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced3f01-9625-4116-992d-d3a5861d145a service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:51.000227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1242/2461] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:53.086083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c10232-8f46-4368-b9dc-8aecb1a424e7 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:53.159056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c10232-8f46-4368-b9dc-8aecb1a424e7 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:53.165134 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:53.165781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:53.166061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:53.166335 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] PATH_INFO: `/v3/users/10a86d22ad7f4362a6b846b296639331` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:53.167557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10a86d22ad7f4362a6b846b296639331)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:53.169526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:53.192812 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:53.220771 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10a86d22ad7f4362a6b846b296639331'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:53.230242 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10a86d22ad7f4362a6b846b296639331'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:53.230950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10a86d22ad7f4362a6b846b296639331'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:53.231762 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10a86d22ad7f4362a6b846b296639331'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:53.250420 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10a86d22ad7f4362a6b846b296639331'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:53.267045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1220/2462] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:53 2022] DELETE /identity/v3/users/10a86d22ad7f4362a6b846b296639331 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:53.272174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a284e4d-c569-49da-ab04-350c116cc045 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:53.272656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a284e4d-c569-49da-ab04-350c116cc045 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:53.272742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a284e4d-c569-49da-ab04-350c116cc045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:53.289010 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:53.297920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-9a284e4d-c569-49da-ab04-350c116cc045 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:53.299492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9a284e4d-c569-49da-ab04-350c116cc045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:53.340633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:53.360153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:53.384707 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1243/2463] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:53.399288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a284e4d-c569-49da-ab04-350c116cc045 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:53.401005 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9a284e4d-c569-49da-ab04-350c116cc045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:53.443492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12e80a-a10a-4ce3-88ec-2a6810388e64 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:53.448282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:53.448915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:53.449216 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:53.449499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] PATH_INFO: `/v3/users/c0a5d83f672d4ecdad08e3057bfea6af` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:53.450630 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0a5d83f672d4ecdad08e3057bfea6af)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:53.452566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:53.472414 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a284e4d-c569-49da-ab04-350c116cc045 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:53.476380 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8beb7-6dce-4cc2-bf98-efa21dbcc49e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:53.477511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8beb7-6dce-4cc2-bf98-efa21dbcc49e service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:53.478219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8beb7-6dce-4cc2-bf98-efa21dbcc49e service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:53.479080 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8beb7-6dce-4cc2-bf98-efa21dbcc49e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:53.493013 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8beb7-6dce-4cc2-bf98-efa21dbcc49e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:53.494599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8beb7-6dce-4cc2-bf98-efa21dbcc49e service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:53.506628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1244/2464] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:58:53.512209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0a5d83f672d4ecdad08e3057bfea6af'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:53.522684 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0a5d83f672d4ecdad08e3057bfea6af'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:53.524079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0a5d83f672d4ecdad08e3057bfea6af'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:53.524957 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0a5d83f672d4ecdad08e3057bfea6af'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:53.541752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0a5d83f672d4ecdad08e3057bfea6af'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:53.560490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1221/2465] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:53 2022] DELETE /identity/v3/users/c0a5d83f672d4ecdad08e3057bfea6af => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:53.810903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8beb7-6dce-4cc2-bf98-efa21dbcc49e service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:53.831982 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8beb7-6dce-4cc2-bf98-efa21dbcc49e service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:53.857588 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:53.859899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:53.888738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:53.918976 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:53.920025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:53.920306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:53.920586 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] PATH_INFO: `/v3/projects/27aaaf73d60b41f088318760f1490e76` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:53.925186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27aaaf73d60b41f088318760f1490e76)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:53.927750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:53.946405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b39b1-7cd6-4f3d-8f9a-15b4d47e759a admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:53.951471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:53.952163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:53.952444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:53.952794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] PATH_INFO: `/v3/projects/631c8fcc50a74d14a62749039515b5a9` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:53.953996 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=631c8fcc50a74d14a62749039515b5a9)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:53.956023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:53.977837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27aaaf73d60b41f088318760f1490e76', 'request_id': 'req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:53.980301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:53.985717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27aaaf73d60b41f088318760f1490e76', 'request_id': 'req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:53.994649 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27aaaf73d60b41f088318760f1490e76', 'request_id': 'req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.010695 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] The token cache is being invalidate because project 27aaaf73d60b41f088318760f1490e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:54.010985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.011727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.012572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '631c8fcc50a74d14a62749039515b5a9', 'request_id': 'req-a7edfc8c-a47c-4713-a595-46d0d2f09d86'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.013295 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1245/2466] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:53 2022] DELETE /identity/v3/projects/27aaaf73d60b41f088318760f1490e76 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:54.017883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:54.019256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:54.022689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '631c8fcc50a74d14a62749039515b5a9', 'request_id': 'req-a7edfc8c-a47c-4713-a595-46d0d2f09d86'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.031137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '631c8fcc50a74d14a62749039515b5a9', 'request_id': 'req-a7edfc8c-a47c-4713-a595-46d0d2f09d86'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.043714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] The token cache is being invalidate because project 631c8fcc50a74d14a62749039515b5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:54.044168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.044943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.046461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1222/2467] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:53 2022] DELETE /identity/v3/projects/631c8fcc50a74d14a62749039515b5a9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:54.050470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:54.052554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:54.090149 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e48c3-43a9-48fd-b3d7-24d1d2705170 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:54.115623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edfc8c-a47c-4713-a595-46d0d2f09d86 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:54.120620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:54.121342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:54.121595 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:54.121891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] PATH_INFO: `/v3/users/b013d1979c594747895a91f63d0ad4a5` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:54.123014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b013d1979c594747895a91f63d0ad4a5)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:54.124967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:54.148773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:54.149448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:54.149797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:54.150113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] PATH_INFO: `/v3/users/91129c3447504c119489b0b740d3fc66` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:54.151306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=91129c3447504c119489b0b740d3fc66)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:54.153626 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:54.174794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b013d1979c594747895a91f63d0ad4a5'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.177660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:54.184692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b013d1979c594747895a91f63d0ad4a5'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.185714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b013d1979c594747895a91f63d0ad4a5'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.186699 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b013d1979c594747895a91f63d0ad4a5'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.201199 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b013d1979c594747895a91f63d0ad4a5'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.207482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91129c3447504c119489b0b740d3fc66'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.213925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1223/2468] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:54 2022] DELETE /identity/v3/users/b013d1979c594747895a91f63d0ad4a5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:54.214662 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91129c3447504c119489b0b740d3fc66'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.215351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91129c3447504c119489b0b740d3fc66'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.228183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91129c3447504c119489b0b740d3fc66'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.236164 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91129c3447504c119489b0b740d3fc66'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.243595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1246/2469] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:54 2022] DELETE /identity/v3/users/91129c3447504c119489b0b740d3fc66 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:54.462884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:54.464400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:54.540916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:54.544838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:54.546136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a074e2e-314c-40ba-8e96-2904f971e3fc admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:54.558218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:54.612906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958fc86-6581-4404-bcac-11dbcbed319f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:54.613542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958fc86-6581-4404-bcac-11dbcbed319f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:54.613872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958fc86-6581-4404-bcac-11dbcbed319f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:54.614146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958fc86-6581-4404-bcac-11dbcbed319f admin admin] PATH_INFO: `/v3/projects/04f138fe94424d97a732140e20c61d90` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:54.615377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958fc86-6581-4404-bcac-11dbcbed319f admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f138fe94424d97a732140e20c61d90)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:54.617270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958fc86-6581-4404-bcac-11dbcbed319f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:54.624134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4467e7-63ce-4361-b625-2f43c20c97ed admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:54.630475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:54.631409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:54.631713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:54.631988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] PATH_INFO: `/v3/projects/9d84a70e1ec045868bd4d47f9f9818da` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:54.633291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d84a70e1ec045868bd4d47f9f9818da)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:54.635718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:54.668937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9958fc86-6581-4404-bcac-11dbcbed319f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f138fe94424d97a732140e20c61d90', 'request_id': 'req-9958fc86-6581-4404-bcac-11dbcbed319f'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.674948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9958fc86-6581-4404-bcac-11dbcbed319f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f138fe94424d97a732140e20c61d90', 'request_id': 'req-9958fc86-6581-4404-bcac-11dbcbed319f'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.682190 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9958fc86-6581-4404-bcac-11dbcbed319f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f138fe94424d97a732140e20c61d90', 'request_id': 'req-9958fc86-6581-4404-bcac-11dbcbed319f'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.686908 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d84a70e1ec045868bd4d47f9f9818da', 'request_id': 'req-76b1dee2-bc49-4829-b579-1c0bc01ba484'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.693683 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d84a70e1ec045868bd4d47f9f9818da', 'request_id': 'req-76b1dee2-bc49-4829-b579-1c0bc01ba484'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.694237 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9958fc86-6581-4404-bcac-11dbcbed319f admin admin] The token cache is being invalidate because project 04f138fe94424d97a732140e20c61d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:54.694524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9958fc86-6581-4404-bcac-11dbcbed319f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.695312 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9958fc86-6581-4404-bcac-11dbcbed319f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.696818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1224/2470] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:54 2022] DELETE /identity/v3/projects/04f138fe94424d97a732140e20c61d90 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:54.702031 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605c132-8e99-4c72-8cfe-0017a7997600 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:54.702381 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605c132-8e99-4c72-8cfe-0017a7997600 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:54.702678 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605c132-8e99-4c72-8cfe-0017a7997600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:54.704399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d84a70e1ec045868bd4d47f9f9818da', 'request_id': 'req-76b1dee2-bc49-4829-b579-1c0bc01ba484'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.711859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:54.718140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] The token cache is being invalidate because project 9d84a70e1ec045868bd4d47f9f9818da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:54.718537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.719222 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.720917 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1247/2471] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:54 2022] DELETE /identity/v3/projects/9d84a70e1ec045868bd4d47f9f9818da => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:54.725098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:54.726091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-5605c132-8e99-4c72-8cfe-0017a7997600 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:54.726692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:54.728006 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5605c132-8e99-4c72-8cfe-0017a7997600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:54.794212 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1225/2472] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:54.796397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1dee2-bc49-4829-b579-1c0bc01ba484 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:54.801867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:54.802456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e57da-86e2-4f25-be35-0d6f0cb1571a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:54.802564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:54.802743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e57da-86e2-4f25-be35-0d6f0cb1571a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:54.802863 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:54.803024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e57da-86e2-4f25-be35-0d6f0cb1571a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:54.803334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] PATH_INFO: `/v3/users/da86b79414f64161bc18d7edd47ca499` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:54.804457 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] RBAC: Authorizing `identity:delete_user(user_id=da86b79414f64161bc18d7edd47ca499)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:54.806393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:54.823325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-4d3e57da-86e2-4f25-be35-0d6f0cb1571a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:54.824601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4d3e57da-86e2-4f25-be35-0d6f0cb1571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:54.833383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1226/2473] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:54.837456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e57da-86e2-4f25-be35-0d6f0cb1571a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:54.849005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e57da-86e2-4f25-be35-0d6f0cb1571a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:54.854646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f031d-7280-4b34-9b92-fdb965f58d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:54.855250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f031d-7280-4b34-9b92-fdb965f58d14 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:54.855624 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f031d-7280-4b34-9b92-fdb965f58d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:54.855929 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f031d-7280-4b34-9b92-fdb965f58d14 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:54.856630 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f031d-7280-4b34-9b92-fdb965f58d14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:58:54.856938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f031d-7280-4b34-9b92-fdb965f58d14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:54.858978 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f031d-7280-4b34-9b92-fdb965f58d14 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:54.867444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:54.877297 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da86b79414f64161bc18d7edd47ca499'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.886385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da86b79414f64161bc18d7edd47ca499'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.886656 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1227/2474] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:58:54.887229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da86b79414f64161bc18d7edd47ca499'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.891449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f031d-7280-4b34-9b92-fdb965f58d14 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:54.903210 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da86b79414f64161bc18d7edd47ca499'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.907738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f031d-7280-4b34-9b92-fdb965f58d14 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:54.912013 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da86b79414f64161bc18d7edd47ca499'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:54.914510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc9098-c266-475a-8613-7de5e4e69320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:54.915285 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc9098-c266-475a-8613-7de5e4e69320 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:54.915581 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc9098-c266-475a-8613-7de5e4e69320 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:54.927732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc9098-c266-475a-8613-7de5e4e69320 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:54.927732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc9098-c266-475a-8613-7de5e4e69320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:54.927732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc9098-c266-475a-8613-7de5e4e69320 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:54.932471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1248/2475] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:54 2022] DELETE /identity/v3/users/da86b79414f64161bc18d7edd47ca499 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:54.964654 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1228/2476] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:54 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:58:54.968674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:54.970256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:55.060739 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb503f46-2a82-4f0d-80da-d6f5c9f3448d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.140708 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520a708-f312-41db-b09d-4c5568ded684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.141297 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520a708-f312-41db-b09d-4c5568ded684 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.141578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520a708-f312-41db-b09d-4c5568ded684 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.141906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520a708-f312-41db-b09d-4c5568ded684 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.142585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520a708-f312-41db-b09d-4c5568ded684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.144664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520a708-f312-41db-b09d-4c5568ded684 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.169152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1249/2477] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:54 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:58:55.173408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc9098-c266-475a-8613-7de5e4e69320 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.185011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc9098-c266-475a-8613-7de5e4e69320 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.190098 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb855e43-ee55-4aea-9707-2b84f3221e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.190695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb855e43-ee55-4aea-9707-2b84f3221e96 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.190970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb855e43-ee55-4aea-9707-2b84f3221e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.191244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb855e43-ee55-4aea-9707-2b84f3221e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.191923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb855e43-ee55-4aea-9707-2b84f3221e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.193504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb855e43-ee55-4aea-9707-2b84f3221e96 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.205673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1229/2478] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:58:55.209917 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520a708-f312-41db-b09d-4c5568ded684 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.220411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520a708-f312-41db-b09d-4c5568ded684 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.225662 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881e07c-4dfe-4459-a9f1-290f8ff6b063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.226278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881e07c-4dfe-4459-a9f1-290f8ff6b063 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.226562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881e07c-4dfe-4459-a9f1-290f8ff6b063 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.226834 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881e07c-4dfe-4459-a9f1-290f8ff6b063 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.227476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881e07c-4dfe-4459-a9f1-290f8ff6b063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.229133 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881e07c-4dfe-4459-a9f1-290f8ff6b063 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.244499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1250/2479] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:55.248938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb855e43-ee55-4aea-9707-2b84f3221e96 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.261536 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb855e43-ee55-4aea-9707-2b84f3221e96 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.266719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f321f-a110-476f-8eef-694b2dd3ae8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.267509 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f321f-a110-476f-8eef-694b2dd3ae8e admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.267827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f321f-a110-476f-8eef-694b2dd3ae8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.268105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f321f-a110-476f-8eef-694b2dd3ae8e admin admin] PATH_INFO: `/v3/projects/81b68c3d993044719744d86acfc7b56e/users/3fb9c56a84bf4b6bbe086214668885bb/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.275914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881e07c-4dfe-4459-a9f1-290f8ff6b063 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.284610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9881e07c-4dfe-4459-a9f1-290f8ff6b063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:55.291198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f321f-a110-476f-8eef-694b2dd3ae8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b68c3d993044719744d86acfc7b56e, user_id=3fb9c56a84bf4b6bbe086214668885bb, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.294114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f321f-a110-476f-8eef-694b2dd3ae8e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.305461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881e07c-4dfe-4459-a9f1-290f8ff6b063 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.314786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.315366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1230/2480] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:55 2022] PUT /identity/v3/projects/81b68c3d993044719744d86acfc7b56e/users/3fb9c56a84bf4b6bbe086214668885bb/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:55.316642 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.317446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.317980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966 admin admin] PATH_INFO: `/v3/projects/9a887dd93e464a0e8eec3c2e6b32a814` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.320002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a887dd93e464a0e8eec3c2e6b32a814)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.323734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.327115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0305cd9-7ef0-40b2-bcf2-9746435e0c95 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.327995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0305cd9-7ef0-40b2-bcf2-9746435e0c95 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.329489 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0305cd9-7ef0-40b2-bcf2-9746435e0c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.369185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-c0305cd9-7ef0-40b2-bcf2-9746435e0c95 None None] MFA Rules not processed for user `3fb9c56a84bf4b6bbe086214668885bb`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:55.371141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c0305cd9-7ef0-40b2-bcf2-9746435e0c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:55.389409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:55.403075 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a887dd93e464a0e8eec3c2e6b32a814', 'request_id': 'req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:55.413854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a887dd93e464a0e8eec3c2e6b32a814', 'request_id': 'req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:55.423000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a887dd93e464a0e8eec3c2e6b32a814', 'request_id': 'req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:55.440793 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966 admin admin] The token cache is being invalidate because project 9a887dd93e464a0e8eec3c2e6b32a814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:55.441306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:55.442737 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-59eaa9c4-dbb0-4cd4-aefd-5380a1cfa966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:55.444494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1251/2481] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:55 2022] DELETE /identity/v3/projects/9a887dd93e464a0e8eec3c2e6b32a814 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:55.450508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa8d9-325a-49cc-aa7e-ee2f6cea54e3 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.450822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa8d9-325a-49cc-aa7e-ee2f6cea54e3 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.451135 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa8d9-325a-49cc-aa7e-ee2f6cea54e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.455867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1231/2482] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:55.461049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0305cd9-7ef0-40b2-bcf2-9746435e0c95 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.473583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-79efa8d9-325a-49cc-aa7e-ee2f6cea54e3 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:55.474893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-79efa8d9-325a-49cc-aa7e-ee2f6cea54e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:55.486384 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:55.539558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0305cd9-7ef0-40b2-bcf2-9746435e0c95 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.540948 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1252/2483] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:55.545200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257dd6a-9d14-46ca-9e13-eda99f498a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.545815 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257dd6a-9d14-46ca-9e13-eda99f498a81 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.546095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257dd6a-9d14-46ca-9e13-eda99f498a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.546366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257dd6a-9d14-46ca-9e13-eda99f498a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.546983 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257dd6a-9d14-46ca-9e13-eda99f498a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.548985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb987bf-a03f-4039-8386-4e081779c3c9 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.549274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb987bf-a03f-4039-8386-4e081779c3c9 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.549390 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257dd6a-9d14-46ca-9e13-eda99f498a81 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.549553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb987bf-a03f-4039-8386-4e081779c3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.571782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-bdb987bf-a03f-4039-8386-4e081779c3c9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:55.573220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-bdb987bf-a03f-4039-8386-4e081779c3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:55.578553 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1232/2484] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:55 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:58:55.582455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257dd6a-9d14-46ca-9e13-eda99f498a81 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.584200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8257dd6a-9d14-46ca-9e13-eda99f498a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:55.584473 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1253/2485] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:55.588721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb987bf-a03f-4039-8386-4e081779c3c9 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.598896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb987bf-a03f-4039-8386-4e081779c3c9 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.654750 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257dd6a-9d14-46ca-9e13-eda99f498a81 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.660885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256e589-440f-4e19-95d8-a9e393fcf8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.661679 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256e589-440f-4e19-95d8-a9e393fcf8cc admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.662033 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256e589-440f-4e19-95d8-a9e393fcf8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.662306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256e589-440f-4e19-95d8-a9e393fcf8cc admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.663188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256e589-440f-4e19-95d8-a9e393fcf8cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.663627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ac75a-25f4-432a-9be4-cf2073c6411f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.664268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ac75a-25f4-432a-9be4-cf2073c6411f admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.664548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ac75a-25f4-432a-9be4-cf2073c6411f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.664841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ac75a-25f4-432a-9be4-cf2073c6411f admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.665527 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ac75a-25f4-432a-9be4-cf2073c6411f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:58:55.665876 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ac75a-25f4-432a-9be4-cf2073c6411f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.665983 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256e589-440f-4e19-95d8-a9e393fcf8cc admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.667504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ac75a-25f4-432a-9be4-cf2073c6411f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.681721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1254/2486] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:58:55.685984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ac75a-25f4-432a-9be4-cf2073c6411f admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.697339 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1233/2487] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:55 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:55.698071 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ac75a-25f4-432a-9be4-cf2073c6411f admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.698336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256e589-440f-4e19-95d8-a9e393fcf8cc admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.702853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15acdb-ca99-4950-8987-3e2bb3719b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.703441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15acdb-ca99-4950-8987-3e2bb3719b48 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.703741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15acdb-ca99-4950-8987-3e2bb3719b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.704031 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15acdb-ca99-4950-8987-3e2bb3719b48 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.704714 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15acdb-ca99-4950-8987-3e2bb3719b48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.706611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15acdb-ca99-4950-8987-3e2bb3719b48 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.708525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256e589-440f-4e19-95d8-a9e393fcf8cc admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.713185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d490b-e3f0-4510-9187-611fbaa1741c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.713766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d490b-e3f0-4510-9187-611fbaa1741c admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.714040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d490b-e3f0-4510-9187-611fbaa1741c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.714306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d490b-e3f0-4510-9187-611fbaa1741c admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.714935 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d490b-e3f0-4510-9187-611fbaa1741c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.716879 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d490b-e3f0-4510-9187-611fbaa1741c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.729444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1234/2488] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:55.733788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d490b-e3f0-4510-9187-611fbaa1741c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.736931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1255/2489] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:55 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:55.740977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15acdb-ca99-4950-8987-3e2bb3719b48 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.747830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d490b-e3f0-4510-9187-611fbaa1741c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.777315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:55.811073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15acdb-ca99-4950-8987-3e2bb3719b48 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.813642 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202a1b4-be1d-4766-a547-099cd6f49c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.814545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202a1b4-be1d-4766-a547-099cd6f49c93 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.814859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202a1b4-be1d-4766-a547-099cd6f49c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.815415 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202a1b4-be1d-4766-a547-099cd6f49c93 admin admin] PATH_INFO: `/v3/projects/17a06fc7add84939a6e1d51ad6946746/users/8e91b61bb8cf43e9bb2a88f18ca160f0/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.817050 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7f990-0d67-4dad-be0e-46b2db678d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.817710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7f990-0d67-4dad-be0e-46b2db678d6e admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.818173 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7f990-0d67-4dad-be0e-46b2db678d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.818450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7f990-0d67-4dad-be0e-46b2db678d6e admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.819078 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7f990-0d67-4dad-be0e-46b2db678d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.821056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7f990-0d67-4dad-be0e-46b2db678d6e admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.831461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202a1b4-be1d-4766-a547-099cd6f49c93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a06fc7add84939a6e1d51ad6946746, user_id=8e91b61bb8cf43e9bb2a88f18ca160f0, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.834855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202a1b4-be1d-4766-a547-099cd6f49c93 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.847128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1256/2490] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:55 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:55.849054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1235/2491] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:55 2022] PUT /identity/v3/projects/17a06fc7add84939a6e1d51ad6946746/users/8e91b61bb8cf43e9bb2a88f18ca160f0/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:55.849382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7f990-0d67-4dad-be0e-46b2db678d6e admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.852927 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202a1b4-be1d-4766-a547-099cd6f49c93 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.857915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:55.859569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7f990-0d67-4dad-be0e-46b2db678d6e admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.919192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9c036-8f09-43c6-9873-416a3b2f4e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.920113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9c036-8f09-43c6-9873-416a3b2f4e7b admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.920611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9c036-8f09-43c6-9873-416a3b2f4e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.921064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9c036-8f09-43c6-9873-416a3b2f4e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.921717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9c036-8f09-43c6-9873-416a3b2f4e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.923328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9c036-8f09-43c6-9873-416a3b2f4e7b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.936750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1257/2492] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:55.937062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202a1b4-be1d-4766-a547-099cd6f49c93 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.937855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9c036-8f09-43c6-9873-416a3b2f4e7b admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.940191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed97983-617c-456d-8cd5-61e3cc7400c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.940857 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed97983-617c-456d-8cd5-61e3cc7400c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.941135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed97983-617c-456d-8cd5-61e3cc7400c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.941403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed97983-617c-456d-8cd5-61e3cc7400c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.942626 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed97983-617c-456d-8cd5-61e3cc7400c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.944390 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed97983-617c-456d-8cd5-61e3cc7400c3 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.952033 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9c036-8f09-43c6-9873-416a3b2f4e7b admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.955930 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1236/2493] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:58:55.959810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e5aa6-62b0-47c4-b5bb-d7b60b0e7334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.960489 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e5aa6-62b0-47c4-b5bb-d7b60b0e7334 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.960792 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e5aa6-62b0-47c4-b5bb-d7b60b0e7334 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.960968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed97983-617c-456d-8cd5-61e3cc7400c3 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.961075 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e5aa6-62b0-47c4-b5bb-d7b60b0e7334 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.961743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e5aa6-62b0-47c4-b5bb-d7b60b0e7334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.963370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e5aa6-62b0-47c4-b5bb-d7b60b0e7334 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.970838 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed97983-617c-456d-8cd5-61e3cc7400c3 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.974774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1258/2494] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:58:55.975566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9153b1e-175c-4cc5-9f6d-4cfbaa27470a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:55.976243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9153b1e-175c-4cc5-9f6d-4cfbaa27470a admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:55.976519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9153b1e-175c-4cc5-9f6d-4cfbaa27470a admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:55.976786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9153b1e-175c-4cc5-9f6d-4cfbaa27470a admin admin] PATH_INFO: `/v3/projects/17a06fc7add84939a6e1d51ad6946746/users/8e91b61bb8cf43e9bb2a88f18ca160f0/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:55.978812 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e5aa6-62b0-47c4-b5bb-d7b60b0e7334 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:55.978926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9153b1e-175c-4cc5-9f6d-4cfbaa27470a admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a06fc7add84939a6e1d51ad6946746, user_id=8e91b61bb8cf43e9bb2a88f18ca160f0, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:55.981555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9153b1e-175c-4cc5-9f6d-4cfbaa27470a admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:55.989065 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e5aa6-62b0-47c4-b5bb-d7b60b0e7334 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:55.991979 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1237/2495] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:55 2022] PUT /identity/v3/projects/17a06fc7add84939a6e1d51ad6946746/users/8e91b61bb8cf43e9bb2a88f18ca160f0/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:56.002062 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb9db0-52cb-48b6-981a-b248e3c6c1ec None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:56.002347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb9db0-52cb-48b6-981a-b248e3c6c1ec None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:56.002621 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb9db0-52cb-48b6-981a-b248e3c6c1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:56.032579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-acbb9db0-52cb-48b6-981a-b248e3c6c1ec None None] MFA Rules not processed for user `8e91b61bb8cf43e9bb2a88f18ca160f0`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:56.033820 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-acbb9db0-52cb-48b6-981a-b248e3c6c1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:56.049269 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cef7f-9a89-436d-82c5-7d0a20eeec27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:56.049930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cef7f-9a89-436d-82c5-7d0a20eeec27 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:56.050208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cef7f-9a89-436d-82c5-7d0a20eeec27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:56.050479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cef7f-9a89-436d-82c5-7d0a20eeec27 admin admin] PATH_INFO: `/v3/projects/894d3ef1b03c466a966fe7145c0b73c2/users/83fa80f82714481da3638d829cb9e2f9/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:56.065272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720cef7f-9a89-436d-82c5-7d0a20eeec27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=894d3ef1b03c466a966fe7145c0b73c2, user_id=83fa80f82714481da3638d829cb9e2f9, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:56.068979 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720cef7f-9a89-436d-82c5-7d0a20eeec27 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:56.083236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1259/2496] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:55 2022] PUT /identity/v3/projects/894d3ef1b03c466a966fe7145c0b73c2/users/83fa80f82714481da3638d829cb9e2f9/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:56.094107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1585d5-d7ff-4098-8c6b-70bf4db39a27 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:56.094391 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1585d5-d7ff-4098-8c6b-70bf4db39a27 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:56.094664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1585d5-d7ff-4098-8c6b-70bf4db39a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:56.111695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1238/2497] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:56.122270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3c1585d5-d7ff-4098-8c6b-70bf4db39a27 None None] MFA Rules not processed for user `83fa80f82714481da3638d829cb9e2f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:56.123488 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3c1585d5-d7ff-4098-8c6b-70bf4db39a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:56.124493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb9db0-52cb-48b6-981a-b248e3c6c1ec None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:56.126726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-acbb9db0-52cb-48b6-981a-b248e3c6c1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:56.141626 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:56.192726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1260/2498] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:58:56.194401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb9db0-52cb-48b6-981a-b248e3c6c1ec None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:56.197345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1585d5-d7ff-4098-8c6b-70bf4db39a27 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:56.221318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:56.246640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65523ff3-b9c2-43f4-abaa-123a9108ede0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:56.247261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65523ff3-b9c2-43f4-abaa-123a9108ede0 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:56.247541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65523ff3-b9c2-43f4-abaa-123a9108ede0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:56.247839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65523ff3-b9c2-43f4-abaa-123a9108ede0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:56.249617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-65523ff3-b9c2-43f4-abaa-123a9108ede0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:56.262459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1585d5-d7ff-4098-8c6b-70bf4db39a27 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:56.267363 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29669f-94fc-416b-bb24-c0b49a5db724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:56.268127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29669f-94fc-416b-bb24-c0b49a5db724 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:56.268405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29669f-94fc-416b-bb24-c0b49a5db724 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:56.268675 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29669f-94fc-416b-bb24-c0b49a5db724 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:56.269306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29669f-94fc-416b-bb24-c0b49a5db724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:56.271260 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29669f-94fc-416b-bb24-c0b49a5db724 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:56.302819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1261/2499] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:56 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:56.303858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29669f-94fc-416b-bb24-c0b49a5db724 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:56.310719 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65523ff3-b9c2-43f4-abaa-123a9108ede0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:56.312355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65523ff3-b9c2-43f4-abaa-123a9108ede0 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:56.321778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1239/2500] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:58:56.365362 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29669f-94fc-416b-bb24-c0b49a5db724 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:56.370062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947d101-0692-4154-ba23-21571b7205ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:56.370654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947d101-0692-4154-ba23-21571b7205ff admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:56.370944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947d101-0692-4154-ba23-21571b7205ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:56.371198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947d101-0692-4154-ba23-21571b7205ff admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:56.371838 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947d101-0692-4154-ba23-21571b7205ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:56.373751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947d101-0692-4154-ba23-21571b7205ff admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:56.397502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1262/2501] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:56 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:56.401541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65523ff3-b9c2-43f4-abaa-123a9108ede0 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:56.414059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65523ff3-b9c2-43f4-abaa-123a9108ede0 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:56.420287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e73812-809c-44fd-946f-7745277e3291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:56.421047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e73812-809c-44fd-946f-7745277e3291 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:56.421331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e73812-809c-44fd-946f-7745277e3291 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:56.421791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e73812-809c-44fd-946f-7745277e3291 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:56.422823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e73812-809c-44fd-946f-7745277e3291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:56.424734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e73812-809c-44fd-946f-7745277e3291 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:56.436709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1240/2502] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:58:56.441174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947d101-0692-4154-ba23-21571b7205ff admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:56.452598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947d101-0692-4154-ba23-21571b7205ff admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:56.458230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1b399-5b9b-47e3-8c62-f6981694f225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:56.458897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1b399-5b9b-47e3-8c62-f6981694f225 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:56.459207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1b399-5b9b-47e3-8c62-f6981694f225 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:56.459525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1b399-5b9b-47e3-8c62-f6981694f225 admin admin] PATH_INFO: `/v3/projects/0fa84a22af9547678024f394b2603666/users/77a34005128f48e38f3c5c01ed2625ff/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:56.476278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1b399-5b9b-47e3-8c62-f6981694f225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fa84a22af9547678024f394b2603666, user_id=77a34005128f48e38f3c5c01ed2625ff, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:56.479177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1b399-5b9b-47e3-8c62-f6981694f225 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:56.496888 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1263/2503] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:56 2022] PUT /identity/v3/projects/0fa84a22af9547678024f394b2603666/users/77a34005128f48e38f3c5c01ed2625ff/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:56.501132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e73812-809c-44fd-946f-7745277e3291 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:56.514267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e73812-809c-44fd-946f-7745277e3291 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:56.593871 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b878339-e455-49c1-9f30-848e01009d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:56.594522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b878339-e455-49c1-9f30-848e01009d42 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:56.594899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b878339-e455-49c1-9f30-848e01009d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:56.595912 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b878339-e455-49c1-9f30-848e01009d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:56.596664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b878339-e455-49c1-9f30-848e01009d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:56.599186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b878339-e455-49c1-9f30-848e01009d42 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:56.609788 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1241/2504] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:56.614075 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1b399-5b9b-47e3-8c62-f6981694f225 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:56.629957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1b399-5b9b-47e3-8c62-f6981694f225 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:56.639321 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faac09a-746a-4e48-9162-1117da351bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:56.640142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faac09a-746a-4e48-9162-1117da351bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:56.640445 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faac09a-746a-4e48-9162-1117da351bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:56.640752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faac09a-746a-4e48-9162-1117da351bfd admin admin] PATH_INFO: `/v3/projects/0fa84a22af9547678024f394b2603666/users/77a34005128f48e38f3c5c01ed2625ff/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:56.644019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faac09a-746a-4e48-9162-1117da351bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fa84a22af9547678024f394b2603666, user_id=77a34005128f48e38f3c5c01ed2625ff, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:56.648664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faac09a-746a-4e48-9162-1117da351bfd admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:56.654097 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:56.704972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1264/2505] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:56 2022] PUT /identity/v3/projects/0fa84a22af9547678024f394b2603666/users/77a34005128f48e38f3c5c01ed2625ff/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:56.709245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc2d7a-8f96-4ade-a694-7abab3c734f9 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:56.709580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc2d7a-8f96-4ade-a694-7abab3c734f9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:56.709867 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc2d7a-8f96-4ade-a694-7abab3c734f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:56.756490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-7fcc2d7a-8f96-4ade-a694-7abab3c734f9 None None] MFA Rules not processed for user `77a34005128f48e38f3c5c01ed2625ff`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:56.757931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7fcc2d7a-8f96-4ade-a694-7abab3c734f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:56.821249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:56.833476 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1242/2506] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:56.845471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faac09a-746a-4e48-9162-1117da351bfd admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:56.903110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faac09a-746a-4e48-9162-1117da351bfd admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:56.907003 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82ebb6-d6e9-40fc-b492-97bed50b244e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:56.907732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82ebb6-d6e9-40fc-b492-97bed50b244e service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:56.908016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82ebb6-d6e9-40fc-b492-97bed50b244e service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:56.908291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82ebb6-d6e9-40fc-b492-97bed50b244e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:56.918404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82ebb6-d6e9-40fc-b492-97bed50b244e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:56.919993 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82ebb6-d6e9-40fc-b492-97bed50b244e service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:56.930691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1265/2507] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:57.021340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc2d7a-8f96-4ade-a694-7abab3c734f9 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:57.030845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc2d7a-8f96-4ade-a694-7abab3c734f9 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:57.034467 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24048ca0-48cf-494b-b8ee-ed814f7a5beb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:57.035057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048ca0-48cf-494b-b8ee-ed814f7a5beb service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:57.035363 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048ca0-48cf-494b-b8ee-ed814f7a5beb service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:57.035657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048ca0-48cf-494b-b8ee-ed814f7a5beb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:57.094232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24048ca0-48cf-494b-b8ee-ed814f7a5beb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:57.095669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24048ca0-48cf-494b-b8ee-ed814f7a5beb service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:57.104628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1243/2508] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:57.524540 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82ebb6-d6e9-40fc-b492-97bed50b244e service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:57.585752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82ebb6-d6e9-40fc-b492-97bed50b244e service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:57.590168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:57.590780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:57.591056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:57.591325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] PATH_INFO: `/v3/users/83fa80f82714481da3638d829cb9e2f9` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:57.592459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83fa80f82714481da3638d829cb9e2f9)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:57.594357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:57.642613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83fa80f82714481da3638d829cb9e2f9'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:57.650170 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83fa80f82714481da3638d829cb9e2f9'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:57.650834 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83fa80f82714481da3638d829cb9e2f9'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:57.662805 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83fa80f82714481da3638d829cb9e2f9'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:57.669883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83fa80f82714481da3638d829cb9e2f9'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:57.675668 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1266/2509] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:57 2022] DELETE /identity/v3/users/83fa80f82714481da3638d829cb9e2f9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:57.680231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e73c3-d8ba-4745-9e3c-f320b51063df None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:57.680726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e73c3-d8ba-4745-9e3c-f320b51063df None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:57.680825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e73c3-d8ba-4745-9e3c-f320b51063df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:57.701275 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-2c7e73c3-d8ba-4745-9e3c-f320b51063df None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:57.702471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2c7e73c3-d8ba-4745-9e3c-f320b51063df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:57.764395 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1244/2510] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:57.774705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:57.776169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:57.798201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:57.848321 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754e4e8-60db-4ba2-a276-8be83b733024 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:57.852000 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bcee0-121c-4145-9ca9-9ddbfec0d775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:57.852603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bcee0-121c-4145-9ca9-9ddbfec0d775 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:57.852914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bcee0-121c-4145-9ca9-9ddbfec0d775 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:57.853193 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bcee0-121c-4145-9ca9-9ddbfec0d775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:57.866202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bcee0-121c-4145-9ca9-9ddbfec0d775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:57.867647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bcee0-121c-4145-9ca9-9ddbfec0d775 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:57.878044 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1267/2511] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:58:58.209246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e73c3-d8ba-4745-9e3c-f320b51063df None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:58.211151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2c7e73c3-d8ba-4745-9e3c-f320b51063df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:58.228330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e73c3-d8ba-4745-9e3c-f320b51063df None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:58.236831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:58.237895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:58.238175 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:58.238444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] PATH_INFO: `/v3/projects/894d3ef1b03c466a966fe7145c0b73c2` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:58.253288 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=894d3ef1b03c466a966fe7145c0b73c2)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:58.256386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:58.305877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '894d3ef1b03c466a966fe7145c0b73c2', 'request_id': 'req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.311146 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '894d3ef1b03c466a966fe7145c0b73c2', 'request_id': 'req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.319385 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '894d3ef1b03c466a966fe7145c0b73c2', 'request_id': 'req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.331348 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] The token cache is being invalidate because project 894d3ef1b03c466a966fe7145c0b73c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:58.331640 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.332346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.334481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1245/2512] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:58 2022] DELETE /identity/v3/projects/894d3ef1b03c466a966fe7145c0b73c2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:58.338669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bcee0-121c-4145-9ca9-9ddbfec0d775 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:58.348462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bcee0-121c-4145-9ca9-9ddbfec0d775 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:58.404488 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5708e2c-2995-4c3c-b80e-4bb1594810a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:58.405106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5708e2c-2995-4c3c-b80e-4bb1594810a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:58.405378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5708e2c-2995-4c3c-b80e-4bb1594810a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:58.405649 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5708e2c-2995-4c3c-b80e-4bb1594810a6 admin admin] PATH_INFO: `/v3/users/77a34005128f48e38f3c5c01ed2625ff` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:58.406753 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5708e2c-2995-4c3c-b80e-4bb1594810a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a34005128f48e38f3c5c01ed2625ff)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:58.408725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5708e2c-2995-4c3c-b80e-4bb1594810a6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:58.453497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e5708e2c-2995-4c3c-b80e-4bb1594810a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a34005128f48e38f3c5c01ed2625ff'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.461311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e5708e2c-2995-4c3c-b80e-4bb1594810a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a34005128f48e38f3c5c01ed2625ff'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.461994 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e5708e2c-2995-4c3c-b80e-4bb1594810a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a34005128f48e38f3c5c01ed2625ff'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.473826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e5708e2c-2995-4c3c-b80e-4bb1594810a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a34005128f48e38f3c5c01ed2625ff'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.481316 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e5708e2c-2995-4c3c-b80e-4bb1594810a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a34005128f48e38f3c5c01ed2625ff'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.489155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1268/2513] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:58:58 2022] DELETE /identity/v3/users/77a34005128f48e38f3c5c01ed2625ff => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:58.763919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:58.765566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:58.778522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbbf83-ad9a-48c3-bb9a-11e1af7af8e0 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:58.790380 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:58.837245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:58.837872 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:58.838148 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:58.838418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1 admin admin] PATH_INFO: `/v3/projects/0fa84a22af9547678024f394b2603666` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:58.839516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fa84a22af9547678024f394b2603666)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:58.841443 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:58.886136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fa84a22af9547678024f394b2603666', 'request_id': 'req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.891620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fa84a22af9547678024f394b2603666', 'request_id': 'req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.898894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fa84a22af9547678024f394b2603666', 'request_id': 'req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.909892 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1 admin admin] The token cache is being invalidate because project 0fa84a22af9547678024f394b2603666 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:58:58.910181 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.911309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2b5bc623-5d4b-47c7-8571-bfcd5849cea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:58:58.912827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1246/2514] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:58:58 2022] DELETE /identity/v3/projects/0fa84a22af9547678024f394b2603666 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:58:58.919211 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848664b-c682-40f0-880b-72be61b09fba None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:58.919748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848664b-c682-40f0-880b-72be61b09fba None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:58.919865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848664b-c682-40f0-880b-72be61b09fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:58.940268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-e848664b-c682-40f0-880b-72be61b09fba None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:58.941617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e848664b-c682-40f0-880b-72be61b09fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:58.975121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:59.003029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1269/2515] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:59.011532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0f3e1-9a14-4a85-89ec-f219a4f964cd None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:59.011847 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0f3e1-9a14-4a85-89ec-f219a4f964cd None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:59.012123 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0f3e1-9a14-4a85-89ec-f219a4f964cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:59.033139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-86a0f3e1-9a14-4a85-89ec-f219a4f964cd None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:59.034334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-86a0f3e1-9a14-4a85-89ec-f219a4f964cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:59.042287 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1247/2516] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:59.046693 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848664b-c682-40f0-880b-72be61b09fba None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:59.056539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848664b-c682-40f0-880b-72be61b09fba None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:59.061574 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51b7f3-c167-4006-b5e2-120e9c25e8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:59.062166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51b7f3-c167-4006-b5e2-120e9c25e8ac admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:59.062443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51b7f3-c167-4006-b5e2-120e9c25e8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:59.062715 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51b7f3-c167-4006-b5e2-120e9c25e8ac admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:59.063356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51b7f3-c167-4006-b5e2-120e9c25e8ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:58:59.063785 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51b7f3-c167-4006-b5e2-120e9c25e8ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:59.065393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51b7f3-c167-4006-b5e2-120e9c25e8ac admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:59.077599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1270/2517] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:58:59.082005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0f3e1-9a14-4a85-89ec-f219a4f964cd None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:59.091653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0f3e1-9a14-4a85-89ec-f219a4f964cd None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:59.096275 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7971c5-66b0-4c98-8fea-7fae2bf20358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:59.096859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7971c5-66b0-4c98-8fea-7fae2bf20358 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:59.097134 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7971c5-66b0-4c98-8fea-7fae2bf20358 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:59.097404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7971c5-66b0-4c98-8fea-7fae2bf20358 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:59.098027 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7971c5-66b0-4c98-8fea-7fae2bf20358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:59.099950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7971c5-66b0-4c98-8fea-7fae2bf20358 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:59.125825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1248/2518] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:58:59 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:58:59.130275 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51b7f3-c167-4006-b5e2-120e9c25e8ac admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:59.140595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51b7f3-c167-4006-b5e2-120e9c25e8ac admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:59.198554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d48816-1969-4659-9310-58c83d04e508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:59.199189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d48816-1969-4659-9310-58c83d04e508 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:59.199472 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d48816-1969-4659-9310-58c83d04e508 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:59.199785 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d48816-1969-4659-9310-58c83d04e508 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:59.200423 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d48816-1969-4659-9310-58c83d04e508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:59.202348 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d48816-1969-4659-9310-58c83d04e508 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:59.228130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1271/2519] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:58:59 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:58:59.232353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7971c5-66b0-4c98-8fea-7fae2bf20358 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:59.243580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7971c5-66b0-4c98-8fea-7fae2bf20358 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:59.248544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee43000-a5c4-4192-be38-8285b5a5171f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:59.249326 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee43000-a5c4-4192-be38-8285b5a5171f admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:59.249509 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee43000-a5c4-4192-be38-8285b5a5171f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:59.249896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee43000-a5c4-4192-be38-8285b5a5171f admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:59.250704 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee43000-a5c4-4192-be38-8285b5a5171f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:59.252423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee43000-a5c4-4192-be38-8285b5a5171f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:59.265041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1249/2520] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:59.268684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d48816-1969-4659-9310-58c83d04e508 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:59.279424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d48816-1969-4659-9310-58c83d04e508 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:59.284567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e6851-965e-4279-8eb8-98a8e5cffaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:59.285168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e6851-965e-4279-8eb8-98a8e5cffaeb admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:59.285486 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e6851-965e-4279-8eb8-98a8e5cffaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:59.286285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e6851-965e-4279-8eb8-98a8e5cffaeb admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:59.286922 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e6851-965e-4279-8eb8-98a8e5cffaeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:59.288552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e6851-965e-4279-8eb8-98a8e5cffaeb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:59.300382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1272/2521] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:58:59.304481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee43000-a5c4-4192-be38-8285b5a5171f admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:59.318400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee43000-a5c4-4192-be38-8285b5a5171f admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:59.319790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5512c8e-d661-48dc-bffe-ac84f624acfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:59.320568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5512c8e-d661-48dc-bffe-ac84f624acfb admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:59.320912 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5512c8e-d661-48dc-bffe-ac84f624acfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:59.321291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5512c8e-d661-48dc-bffe-ac84f624acfb admin admin] PATH_INFO: `/v3/projects/c12ac892aa6b4154b90e992217298624/users/0bd506975b28441cae07cd3bceb04970/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:59.342278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5512c8e-d661-48dc-bffe-ac84f624acfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12ac892aa6b4154b90e992217298624, user_id=0bd506975b28441cae07cd3bceb04970, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:59.345055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5512c8e-d661-48dc-bffe-ac84f624acfb admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:59.361012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1250/2522] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:58:59 2022] PUT /identity/v3/projects/c12ac892aa6b4154b90e992217298624/users/0bd506975b28441cae07cd3bceb04970/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:58:59.372678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d673e31-1368-4f1c-af2f-d352ba36a26f None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:59.372966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d673e31-1368-4f1c-af2f-d352ba36a26f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:59.373235 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d673e31-1368-4f1c-af2f-d352ba36a26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:59.404034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-9d673e31-1368-4f1c-af2f-d352ba36a26f None None] MFA Rules not processed for user `0bd506975b28441cae07cd3bceb04970`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:58:59.405705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9d673e31-1368-4f1c-af2f-d352ba36a26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:59.485604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1273/2523] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:58:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:58:59.568583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5512c8e-d661-48dc-bffe-ac84f624acfb admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:58:59.570479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e5512c8e-d661-48dc-bffe-ac84f624acfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:58:59.637332 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:58:59.668516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5512c8e-d661-48dc-bffe-ac84f624acfb admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:58:59.672324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed06cb-faf1-4dc6-8f7e-28f879e89d32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:58:59.673230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed06cb-faf1-4dc6-8f7e-28f879e89d32 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:58:59.673230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed06cb-faf1-4dc6-8f7e-28f879e89d32 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:58:59.673462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed06cb-faf1-4dc6-8f7e-28f879e89d32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:58:59.684292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed06cb-faf1-4dc6-8f7e-28f879e89d32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:58:59.686388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed06cb-faf1-4dc6-8f7e-28f879e89d32 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:58:59.699865 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1251/2524] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:58:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:00.781067 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d673e31-1368-4f1c-af2f-d352ba36a26f None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:00.782568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9d673e31-1368-4f1c-af2f-d352ba36a26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:00.796234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d673e31-1368-4f1c-af2f-d352ba36a26f None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:00.799940 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2bc362-2976-4630-9af6-dede8d964d2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:00.800548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bc362-2976-4630-9af6-dede8d964d2d service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:00.800829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bc362-2976-4630-9af6-dede8d964d2d service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:00.801104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bc362-2976-4630-9af6-dede8d964d2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:00.802918 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6c2bc362-2976-4630-9af6-dede8d964d2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:00.837286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:00.870823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2bc362-2976-4630-9af6-dede8d964d2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:00.872324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2bc362-2976-4630-9af6-dede8d964d2d service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:00.882374 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1274/2525] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:01.123076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a291ba3-31a7-4281-8569-c4e789ad04ee None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:01.123655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a291ba3-31a7-4281-8569-c4e789ad04ee None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:01.123655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a291ba3-31a7-4281-8569-c4e789ad04ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:01.148792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-9a291ba3-31a7-4281-8569-c4e789ad04ee None None] MFA Rules not processed for user `0bd506975b28441cae07cd3bceb04970`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:01.150364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9a291ba3-31a7-4281-8569-c4e789ad04ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:01.161523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1252/2526] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:05.898057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2bc362-2976-4630-9af6-dede8d964d2d service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:05.898057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6c2bc362-2976-4630-9af6-dede8d964d2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:05.972623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2bc362-2976-4630-9af6-dede8d964d2d service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:05.977238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:05.977926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:05.978206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:05.978481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] PATH_INFO: `/v3/users/3fb9c56a84bf4b6bbe086214668885bb` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:05.979629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fb9c56a84bf4b6bbe086214668885bb)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:05.981607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:06.029397 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fb9c56a84bf4b6bbe086214668885bb'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.036233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fb9c56a84bf4b6bbe086214668885bb'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.036951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fb9c56a84bf4b6bbe086214668885bb'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.049368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fb9c56a84bf4b6bbe086214668885bb'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.056997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fb9c56a84bf4b6bbe086214668885bb'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.063289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1275/2527] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:05 2022] DELETE /identity/v3/users/3fb9c56a84bf4b6bbe086214668885bb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:06.068392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce1597-5ef1-4101-9257-7933335c8e39 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:06.068886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce1597-5ef1-4101-9257-7933335c8e39 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:06.069093 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce1597-5ef1-4101-9257-7933335c8e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:06.088651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-86ce1597-5ef1-4101-9257-7933335c8e39 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:06.090061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-86ce1597-5ef1-4101-9257-7933335c8e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:06.151448 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1253/2528] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:06.160930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:06.162355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:06.220885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:06.228342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ad8b9-3629-4c9a-91a9-e9162cc03ce1 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:06.235836 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d413e-276e-4b88-bc69-229ff03da6ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:06.237046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d413e-276e-4b88-bc69-229ff03da6ca service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:06.237286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d413e-276e-4b88-bc69-229ff03da6ca service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:06.237620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d413e-276e-4b88-bc69-229ff03da6ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:06.252857 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d413e-276e-4b88-bc69-229ff03da6ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:06.255649 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d413e-276e-4b88-bc69-229ff03da6ca service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:06.266683 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1276/2529] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:06.527578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce1597-5ef1-4101-9257-7933335c8e39 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:06.530001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-86ce1597-5ef1-4101-9257-7933335c8e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:06.549651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce1597-5ef1-4101-9257-7933335c8e39 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:06.556370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3823a-9238-4cf2-a703-447e0aae9df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:06.557185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3823a-9238-4cf2-a703-447e0aae9df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:06.557559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3823a-9238-4cf2-a703-447e0aae9df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:06.557928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3823a-9238-4cf2-a703-447e0aae9df1 admin admin] PATH_INFO: `/v3/users/a3226c1153fb43ccbf3eb0a9749b7f7c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:06.559360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3823a-9238-4cf2-a703-447e0aae9df1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3226c1153fb43ccbf3eb0a9749b7f7c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:06.562193 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3823a-9238-4cf2-a703-447e0aae9df1 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:06.602539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d413e-276e-4b88-bc69-229ff03da6ca service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:06.602539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d10d413e-276e-4b88-bc69-229ff03da6ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:06.621376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d413e-276e-4b88-bc69-229ff03da6ca service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:06.626799 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:06.627723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:06.628056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:06.628419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] PATH_INFO: `/v3/projects/81b68c3d993044719744d86acfc7b56e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:06.629738 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81b68c3d993044719744d86acfc7b56e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:06.631946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:06.663174 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-66b3823a-9238-4cf2-a703-447e0aae9df1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3226c1153fb43ccbf3eb0a9749b7f7c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.670539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-66b3823a-9238-4cf2-a703-447e0aae9df1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3226c1153fb43ccbf3eb0a9749b7f7c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.672162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-66b3823a-9238-4cf2-a703-447e0aae9df1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3226c1153fb43ccbf3eb0a9749b7f7c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.672866 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-66b3823a-9238-4cf2-a703-447e0aae9df1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3226c1153fb43ccbf3eb0a9749b7f7c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.691857 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-66b3823a-9238-4cf2-a703-447e0aae9df1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3226c1153fb43ccbf3eb0a9749b7f7c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.720803 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1254/2530] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:06 2022] DELETE /identity/v3/users/a3226c1153fb43ccbf3eb0a9749b7f7c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:06.725670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a943747-b1e2-42c8-80a8-3298a8ca65db None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:06.726532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a943747-b1e2-42c8-80a8-3298a8ca65db None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:06.727090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a943747-b1e2-42c8-80a8-3298a8ca65db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:06.737386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81b68c3d993044719744d86acfc7b56e', 'request_id': 'req-83f1d948-ddae-41be-ae0a-0095468708e8'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.753835 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81b68c3d993044719744d86acfc7b56e', 'request_id': 'req-83f1d948-ddae-41be-ae0a-0095468708e8'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.760499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-6a943747-b1e2-42c8-80a8-3298a8ca65db None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:06.762303 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6a943747-b1e2-42c8-80a8-3298a8ca65db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:06.767030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81b68c3d993044719744d86acfc7b56e', 'request_id': 'req-83f1d948-ddae-41be-ae0a-0095468708e8'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.784334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] The token cache is being invalidate because project 81b68c3d993044719744d86acfc7b56e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:06.785032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.786638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.789377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1277/2531] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:06 2022] DELETE /identity/v3/projects/81b68c3d993044719744d86acfc7b56e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:06.794233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:06.795848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:06.836756 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:06.858734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1255/2532] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:06.868652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a943747-b1e2-42c8-80a8-3298a8ca65db None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:06.870247 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6a943747-b1e2-42c8-80a8-3298a8ca65db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:06.882862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1d948-ddae-41be-ae0a-0095468708e8 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:06.889515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:06.890398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:06.890833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:06.891291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] PATH_INFO: `/v3/users/8e91b61bb8cf43e9bb2a88f18ca160f0` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:06.892508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e91b61bb8cf43e9bb2a88f18ca160f0)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:06.894471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:06.943831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a943747-b1e2-42c8-80a8-3298a8ca65db None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:06.948259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47e1ca-c66e-4a4c-80ab-cdee085649b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:06.948783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47e1ca-c66e-4a4c-80ab-cdee085649b7 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:06.949355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47e1ca-c66e-4a4c-80ab-cdee085649b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:06.949565 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47e1ca-c66e-4a4c-80ab-cdee085649b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:06.951205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e91b61bb8cf43e9bb2a88f18ca160f0'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.962058 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47e1ca-c66e-4a4c-80ab-cdee085649b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:06.963448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e91b61bb8cf43e9bb2a88f18ca160f0'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.964698 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47e1ca-c66e-4a4c-80ab-cdee085649b7 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:06.965090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e91b61bb8cf43e9bb2a88f18ca160f0'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.983866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e91b61bb8cf43e9bb2a88f18ca160f0'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:06.990358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:06.996394 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1256/2533] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:06.999385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e91b61bb8cf43e9bb2a88f18ca160f0'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.008098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1278/2534] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:06 2022] DELETE /identity/v3/users/8e91b61bb8cf43e9bb2a88f18ca160f0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:07.366931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47e1ca-c66e-4a4c-80ab-cdee085649b7 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:07.368564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-db47e1ca-c66e-4a4c-80ab-cdee085649b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:07.392152 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47e1ca-c66e-4a4c-80ab-cdee085649b7 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:07.414956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:07.416943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:07.473227 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:07.473915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:07.474199 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:07.474530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] PATH_INFO: `/v3/projects/ed8db0e68b354b739f071af29760345a` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:07.475779 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed8db0e68b354b739f071af29760345a)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:07.477939 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:07.500686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5ee6f-e337-4576-989c-0e89ecacde75 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:07.505407 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:07.506036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:07.506350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:07.506654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3 admin admin] PATH_INFO: `/v3/projects/17a06fc7add84939a6e1d51ad6946746` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:07.509098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17a06fc7add84939a6e1d51ad6946746)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:07.511843 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:07.529302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed8db0e68b354b739f071af29760345a', 'request_id': 'req-dac1ca2a-fcbe-43c6-acba-cef009d74208'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.536046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed8db0e68b354b739f071af29760345a', 'request_id': 'req-dac1ca2a-fcbe-43c6-acba-cef009d74208'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.544665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed8db0e68b354b739f071af29760345a', 'request_id': 'req-dac1ca2a-fcbe-43c6-acba-cef009d74208'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.556873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] The token cache is being invalidate because project ed8db0e68b354b739f071af29760345a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:07.557157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.557799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.559311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1257/2535] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:07 2022] DELETE /identity/v3/projects/ed8db0e68b354b739f071af29760345a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:07.561251 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17a06fc7add84939a6e1d51ad6946746', 'request_id': 'req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.563291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:07.564721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:07.568294 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17a06fc7add84939a6e1d51ad6946746', 'request_id': 'req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.576171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17a06fc7add84939a6e1d51ad6946746', 'request_id': 'req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.588815 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3 admin admin] The token cache is being invalidate because project 17a06fc7add84939a6e1d51ad6946746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:07.589108 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.589762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-16e6172d-ce05-4ad0-b158-78ef0b57f9b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.591264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1279/2536] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:07 2022] DELETE /identity/v3/projects/17a06fc7add84939a6e1d51ad6946746 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:07.593389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:07.598706 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848480f-78fb-43e4-ba7a-2be0d6cd19cc None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:07.598995 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848480f-78fb-43e4-ba7a-2be0d6cd19cc None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:07.599237 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848480f-78fb-43e4-ba7a-2be0d6cd19cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:07.619127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3848480f-78fb-43e4-ba7a-2be0d6cd19cc None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:07.620374 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3848480f-78fb-43e4-ba7a-2be0d6cd19cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:07.633488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1ca2a-fcbe-43c6-acba-cef009d74208 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:07.682236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1280/2537] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:07.692408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1e54f-1d83-470b-aaae-8a9ad2fb1d86 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:07.692678 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1e54f-1d83-470b-aaae-8a9ad2fb1d86 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:07.692956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1e54f-1d83-470b-aaae-8a9ad2fb1d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:07.694269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:07.694916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:07.695198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:07.695670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] PATH_INFO: `/v3/users/dbc7ae2daeea49e49115a587d59bbc30` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:07.696688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbc7ae2daeea49e49115a587d59bbc30)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:07.698817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:07.714314 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-85f1e54f-1d83-470b-aaae-8a9ad2fb1d86 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:07.715538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-85f1e54f-1d83-470b-aaae-8a9ad2fb1d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:07.730069 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1281/2538] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:07.734461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1e54f-1d83-470b-aaae-8a9ad2fb1d86 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:07.745665 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1e54f-1d83-470b-aaae-8a9ad2fb1d86 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:07.752907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ee34d-faff-4edf-b951-0c948bb3c1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:07.753801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee34d-faff-4edf-b951-0c948bb3c1ce admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:07.754122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee34d-faff-4edf-b951-0c948bb3c1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:07.754942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee34d-faff-4edf-b951-0c948bb3c1ce admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:07.755596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ee34d-faff-4edf-b951-0c948bb3c1ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:07.755933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ee34d-faff-4edf-b951-0c948bb3c1ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:07.757533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ee34d-faff-4edf-b951-0c948bb3c1ce admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:07.762071 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbc7ae2daeea49e49115a587d59bbc30'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.775284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1282/2539] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:59:07.776279 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbc7ae2daeea49e49115a587d59bbc30'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.776409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ee34d-faff-4edf-b951-0c948bb3c1ce admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:07.777258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbc7ae2daeea49e49115a587d59bbc30'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.777950 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbc7ae2daeea49e49115a587d59bbc30'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.784686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:07.791036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ee34d-faff-4edf-b951-0c948bb3c1ce admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:07.793010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbc7ae2daeea49e49115a587d59bbc30'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:07.800913 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5dd81-4cba-4703-a9aa-2b57846f6500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:07.801663 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5dd81-4cba-4703-a9aa-2b57846f6500 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:07.801955 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5dd81-4cba-4703-a9aa-2b57846f6500 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:07.802240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5dd81-4cba-4703-a9aa-2b57846f6500 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:07.802926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5dd81-4cba-4703-a9aa-2b57846f6500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:07.805707 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5dd81-4cba-4703-a9aa-2b57846f6500 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:07.808834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1258/2540] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:07 2022] DELETE /identity/v3/users/dbc7ae2daeea49e49115a587d59bbc30 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:07.841864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1283/2541] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:07 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:59:07.843413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:07.845075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:07.925487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:07.936486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4b94f-2b80-48bf-bbac-b5c165fa2306 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:07.999257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15619d0-d21a-4aa6-bf91-5db93983ddd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:08.000088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15619d0-d21a-4aa6-bf91-5db93983ddd7 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.000246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15619d0-d21a-4aa6-bf91-5db93983ddd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.000518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15619d0-d21a-4aa6-bf91-5db93983ddd7 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.001160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15619d0-d21a-4aa6-bf91-5db93983ddd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:08.003106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15619d0-d21a-4aa6-bf91-5db93983ddd7 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:08.027054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1259/2542] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:07 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:08.031345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5dd81-4cba-4703-a9aa-2b57846f6500 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:08.042462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5dd81-4cba-4703-a9aa-2b57846f6500 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:08.047069 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6e784-86b7-484c-a7b4-e9b6497f7120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:08.047674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6e784-86b7-484c-a7b4-e9b6497f7120 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.047988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6e784-86b7-484c-a7b4-e9b6497f7120 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.048265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6e784-86b7-484c-a7b4-e9b6497f7120 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.048915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6e784-86b7-484c-a7b4-e9b6497f7120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:08.050594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6e784-86b7-484c-a7b4-e9b6497f7120 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:08.061919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1284/2543] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:08.065945 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15619d0-d21a-4aa6-bf91-5db93983ddd7 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:08.076742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15619d0-d21a-4aa6-bf91-5db93983ddd7 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:08.082355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400db7a6-3e33-4084-9d53-16069d94bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:08.082972 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400db7a6-3e33-4084-9d53-16069d94bd41 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.083254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400db7a6-3e33-4084-9d53-16069d94bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.083554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400db7a6-3e33-4084-9d53-16069d94bd41 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.084269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400db7a6-3e33-4084-9d53-16069d94bd41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:08.085875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400db7a6-3e33-4084-9d53-16069d94bd41 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:08.096765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1260/2544] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:08.101711 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6e784-86b7-484c-a7b4-e9b6497f7120 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:08.113145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6e784-86b7-484c-a7b4-e9b6497f7120 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:08.119203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a274f-fa32-474e-b0ae-40b813202424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:08.119969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a274f-fa32-474e-b0ae-40b813202424 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.120558 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a274f-fa32-474e-b0ae-40b813202424 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.120826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a274f-fa32-474e-b0ae-40b813202424 admin admin] PATH_INFO: `/v3/projects/499fb4b52df7483cbc30dca81e5f0ce1/users/b96eca838aba40ddb2e5b3c4f52c883d/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.128400 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400db7a6-3e33-4084-9d53-16069d94bd41 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:08.130340 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-400db7a6-3e33-4084-9d53-16069d94bd41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:08.141234 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a274f-fa32-474e-b0ae-40b813202424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=499fb4b52df7483cbc30dca81e5f0ce1, user_id=b96eca838aba40ddb2e5b3c4f52c883d, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:08.144462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a274f-fa32-474e-b0ae-40b813202424 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:08.159259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1285/2545] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:08 2022] PUT /identity/v3/projects/499fb4b52df7483cbc30dca81e5f0ce1/users/b96eca838aba40ddb2e5b3c4f52c883d/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:08.165226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400db7a6-3e33-4084-9d53-16069d94bd41 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:08.171148 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500e465-288a-471d-a8ee-8431c1f2b559 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.171437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500e465-288a-471d-a8ee-8431c1f2b559 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.171756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500e465-288a-471d-a8ee-8431c1f2b559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.201054 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-a500e465-288a-471d-a8ee-8431c1f2b559 None None] MFA Rules not processed for user `b96eca838aba40ddb2e5b3c4f52c883d`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:08.202299 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a500e465-288a-471d-a8ee-8431c1f2b559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:08.229289 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090f7e4-c6cf-4498-bdf5-3631c9562919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:08.229935 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090f7e4-c6cf-4498-bdf5-3631c9562919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.230210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090f7e4-c6cf-4498-bdf5-3631c9562919 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.230485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090f7e4-c6cf-4498-bdf5-3631c9562919 admin admin] PATH_INFO: `/v3/projects/60f7c4818b4c41f6ab6777e560397a42` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.231659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090f7e4-c6cf-4498-bdf5-3631c9562919 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60f7c4818b4c41f6ab6777e560397a42)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:08.233596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090f7e4-c6cf-4498-bdf5-3631c9562919 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:08.283893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1286/2546] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:08.295354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500e465-288a-471d-a8ee-8431c1f2b559 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:08.296924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a500e465-288a-471d-a8ee-8431c1f2b559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:08.299245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b090f7e4-c6cf-4498-bdf5-3631c9562919 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60f7c4818b4c41f6ab6777e560397a42', 'request_id': 'req-b090f7e4-c6cf-4498-bdf5-3631c9562919'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:08.306492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b090f7e4-c6cf-4498-bdf5-3631c9562919 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60f7c4818b4c41f6ab6777e560397a42', 'request_id': 'req-b090f7e4-c6cf-4498-bdf5-3631c9562919'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:08.315501 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b090f7e4-c6cf-4498-bdf5-3631c9562919 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60f7c4818b4c41f6ab6777e560397a42', 'request_id': 'req-b090f7e4-c6cf-4498-bdf5-3631c9562919'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:08.339419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b090f7e4-c6cf-4498-bdf5-3631c9562919 admin admin] The token cache is being invalidate because project 60f7c4818b4c41f6ab6777e560397a42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:08.339743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b090f7e4-c6cf-4498-bdf5-3631c9562919 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:08.340818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-b090f7e4-c6cf-4498-bdf5-3631c9562919 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:08.346471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1261/2547] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:08 2022] DELETE /identity/v3/projects/60f7c4818b4c41f6ab6777e560397a42 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:08.348156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c17ca8-a80c-45d9-80d4-1aba9bc03fa6 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.348445 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c17ca8-a80c-45d9-80d4-1aba9bc03fa6 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.348723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c17ca8-a80c-45d9-80d4-1aba9bc03fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.364170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:08.371543 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500e465-288a-471d-a8ee-8431c1f2b559 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:08.372112 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-12c17ca8-a80c-45d9-80d4-1aba9bc03fa6 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:08.374248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-12c17ca8-a80c-45d9-80d4-1aba9bc03fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:08.435195 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb808c8-cc7e-490f-a011-92029a072e87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:08.435937 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb808c8-cc7e-490f-a011-92029a072e87 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.436288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb808c8-cc7e-490f-a011-92029a072e87 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.436624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb808c8-cc7e-490f-a011-92029a072e87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.438493 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-bbb808c8-cc7e-490f-a011-92029a072e87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:08.445346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:08.451705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1262/2548] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:08.459783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc276e00-c633-4e7c-bbf3-dda29f4af49f None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.460228 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc276e00-c633-4e7c-bbf3-dda29f4af49f None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.460508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc276e00-c633-4e7c-bbf3-dda29f4af49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.480882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-cc276e00-c633-4e7c-bbf3-dda29f4af49f None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:08.482121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-cc276e00-c633-4e7c-bbf3-dda29f4af49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:08.491702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1263/2549] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:08.496566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc276e00-c633-4e7c-bbf3-dda29f4af49f None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:08.507409 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc276e00-c633-4e7c-bbf3-dda29f4af49f None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:08.513250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2f55e-4e67-42ab-9d18-b2b59bd9350f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:08.514197 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2f55e-4e67-42ab-9d18-b2b59bd9350f admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.514482 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2f55e-4e67-42ab-9d18-b2b59bd9350f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.514758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2f55e-4e67-42ab-9d18-b2b59bd9350f admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.515398 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2f55e-4e67-42ab-9d18-b2b59bd9350f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:08.516066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2f55e-4e67-42ab-9d18-b2b59bd9350f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:08.517344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2f55e-4e67-42ab-9d18-b2b59bd9350f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:08.518283 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb808c8-cc7e-490f-a011-92029a072e87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:08.520777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb808c8-cc7e-490f-a011-92029a072e87 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:08.533238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1264/2550] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:59:08.533431 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1287/2551] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:08.534665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2f55e-4e67-42ab-9d18-b2b59bd9350f admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:08.547121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2f55e-4e67-42ab-9d18-b2b59bd9350f admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:08.552975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f45142-a1d8-4953-ad54-28e5019e75d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:08.553595 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f45142-a1d8-4953-ad54-28e5019e75d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.553874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f45142-a1d8-4953-ad54-28e5019e75d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.554154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f45142-a1d8-4953-ad54-28e5019e75d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.554792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f45142-a1d8-4953-ad54-28e5019e75d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:08.556966 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f45142-a1d8-4953-ad54-28e5019e75d4 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:08.601667 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1265/2552] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:08 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:08.602857 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb808c8-cc7e-490f-a011-92029a072e87 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:08.617974 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb808c8-cc7e-490f-a011-92029a072e87 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:08.703370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919686f-c680-41c5-a4d1-bc99ab34969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:08.704224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919686f-c680-41c5-a4d1-bc99ab34969c admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.704636 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919686f-c680-41c5-a4d1-bc99ab34969c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.705015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919686f-c680-41c5-a4d1-bc99ab34969c admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.706270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919686f-c680-41c5-a4d1-bc99ab34969c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:08.709087 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919686f-c680-41c5-a4d1-bc99ab34969c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:08.741602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1288/2553] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:08 2022] POST /identity/v3/users => generated 291 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:08.745491 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f45142-a1d8-4953-ad54-28e5019e75d4 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:08.758104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f45142-a1d8-4953-ad54-28e5019e75d4 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:08.764097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e9fef-0ba2-4602-8b65-21be8ea989ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:08.765142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e9fef-0ba2-4602-8b65-21be8ea989ee admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.765524 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e9fef-0ba2-4602-8b65-21be8ea989ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.765849 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e9fef-0ba2-4602-8b65-21be8ea989ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.766817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e9fef-0ba2-4602-8b65-21be8ea989ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:08.769231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e9fef-0ba2-4602-8b65-21be8ea989ee admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:08.785346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1266/2554] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:08.786023 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919686f-c680-41c5-a4d1-bc99ab34969c admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:08.801497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919686f-c680-41c5-a4d1-bc99ab34969c admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:08.812316 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75197c-f474-44d6-858b-083bb02f38d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:08.814375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75197c-f474-44d6-858b-083bb02f38d4 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.815433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75197c-f474-44d6-858b-083bb02f38d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.815742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75197c-f474-44d6-858b-083bb02f38d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.817157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75197c-f474-44d6-858b-083bb02f38d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:08.819482 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75197c-f474-44d6-858b-083bb02f38d4 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:08.860633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1289/2555] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:08.861938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e9fef-0ba2-4602-8b65-21be8ea989ee admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:08.876201 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e9fef-0ba2-4602-8b65-21be8ea989ee admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:08.882764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436b94b-de8f-4f30-ac98-8f3c167bf113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:08.883845 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436b94b-de8f-4f30-ac98-8f3c167bf113 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.884177 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436b94b-de8f-4f30-ac98-8f3c167bf113 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.884460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436b94b-de8f-4f30-ac98-8f3c167bf113 admin admin] PATH_INFO: `/v3/projects/03954171dedd434f882c7d1a48cbb0b8/users/2680099c0a2249449da37557ae58880c/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:08.915880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436b94b-de8f-4f30-ac98-8f3c167bf113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03954171dedd434f882c7d1a48cbb0b8, user_id=2680099c0a2249449da37557ae58880c, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:08.919374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436b94b-de8f-4f30-ac98-8f3c167bf113 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:08.954479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1267/2556] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:08 2022] PUT /identity/v3/projects/03954171dedd434f882c7d1a48cbb0b8/users/2680099c0a2249449da37557ae58880c/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:08.970169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af939e2-ec4d-4dae-adb7-0c8c4d060170 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:08.970592 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af939e2-ec4d-4dae-adb7-0c8c4d060170 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:08.971059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af939e2-ec4d-4dae-adb7-0c8c4d060170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:09.010442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-6af939e2-ec4d-4dae-adb7-0c8c4d060170 None None] MFA Rules not processed for user `2680099c0a2249449da37557ae58880c`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:09.011829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6af939e2-ec4d-4dae-adb7-0c8c4d060170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:09.091410 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1290/2557] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:09.092647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436b94b-de8f-4f30-ac98-8f3c167bf113 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:09.169474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436b94b-de8f-4f30-ac98-8f3c167bf113 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:09.174358 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84a5cc-24b8-4ff4-bd4f-36c7cc1b76e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:09.174959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84a5cc-24b8-4ff4-bd4f-36c7cc1b76e4 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:09.175463 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84a5cc-24b8-4ff4-bd4f-36c7cc1b76e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:09.175571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84a5cc-24b8-4ff4-bd4f-36c7cc1b76e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:09.176255 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84a5cc-24b8-4ff4-bd4f-36c7cc1b76e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:09.178197 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84a5cc-24b8-4ff4-bd4f-36c7cc1b76e4 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:09.207670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1268/2558] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:09 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:09.210761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af939e2-ec4d-4dae-adb7-0c8c4d060170 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:09.222237 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af939e2-ec4d-4dae-adb7-0c8c4d060170 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:09.235067 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:09.287178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a01b7e-f68a-4e9a-a4bb-a363d548635b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:09.287822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a01b7e-f68a-4e9a-a4bb-a363d548635b admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:09.288104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a01b7e-f68a-4e9a-a4bb-a363d548635b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:09.288379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a01b7e-f68a-4e9a-a4bb-a363d548635b admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:09.289018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a01b7e-f68a-4e9a-a4bb-a363d548635b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:09.290996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a01b7e-f68a-4e9a-a4bb-a363d548635b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:09.312497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1291/2559] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:09 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:59:09.316468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84a5cc-24b8-4ff4-bd4f-36c7cc1b76e4 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:09.327936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84a5cc-24b8-4ff4-bd4f-36c7cc1b76e4 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:09.333526 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991df58-4a92-4427-bac6-ad425442e7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:09.334071 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991df58-4a92-4427-bac6-ad425442e7ee admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:09.334347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991df58-4a92-4427-bac6-ad425442e7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:09.334617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991df58-4a92-4427-bac6-ad425442e7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:09.335309 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991df58-4a92-4427-bac6-ad425442e7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:09.336972 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991df58-4a92-4427-bac6-ad425442e7ee admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:09.349135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1269/2560] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:09.353269 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a01b7e-f68a-4e9a-a4bb-a363d548635b admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:09.364291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a01b7e-f68a-4e9a-a4bb-a363d548635b admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:09.372356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be8a8b-2b9e-48f6-8c27-4463aa059953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:09.373336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be8a8b-2b9e-48f6-8c27-4463aa059953 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:09.373997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be8a8b-2b9e-48f6-8c27-4463aa059953 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:09.373997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be8a8b-2b9e-48f6-8c27-4463aa059953 admin admin] PATH_INFO: `/v3/projects/91cd2a8105844171a5f3ab694ab3f8a7/users/212d35a7e33046c59e3bace23a863d2a/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:09.390771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be8a8b-2b9e-48f6-8c27-4463aa059953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cd2a8105844171a5f3ab694ab3f8a7, user_id=212d35a7e33046c59e3bace23a863d2a, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:09.393458 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be8a8b-2b9e-48f6-8c27-4463aa059953 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:09.411775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1292/2561] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:09 2022] PUT /identity/v3/projects/91cd2a8105844171a5f3ab694ab3f8a7/users/212d35a7e33046c59e3bace23a863d2a/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:09.413627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991df58-4a92-4427-bac6-ad425442e7ee admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:09.420302 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:09.424362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991df58-4a92-4427-bac6-ad425442e7ee admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:09.499097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726d81a-389c-4dba-8b90-2b5c601fa9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:09.499866 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726d81a-389c-4dba-8b90-2b5c601fa9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:09.500161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726d81a-389c-4dba-8b90-2b5c601fa9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:09.500438 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726d81a-389c-4dba-8b90-2b5c601fa9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:09.501091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726d81a-389c-4dba-8b90-2b5c601fa9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:09.503108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726d81a-389c-4dba-8b90-2b5c601fa9b2 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:09.519647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1270/2562] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:09.524479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be8a8b-2b9e-48f6-8c27-4463aa059953 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:09.535025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be8a8b-2b9e-48f6-8c27-4463aa059953 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:09.539926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcf091-a119-4443-9faf-2db11df17bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:09.540601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcf091-a119-4443-9faf-2db11df17bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:09.540879 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcf091-a119-4443-9faf-2db11df17bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:09.541197 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcf091-a119-4443-9faf-2db11df17bdb admin admin] PATH_INFO: `/v3/projects/91cd2a8105844171a5f3ab694ab3f8a7/users/212d35a7e33046c59e3bace23a863d2a/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:09.543119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcf091-a119-4443-9faf-2db11df17bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cd2a8105844171a5f3ab694ab3f8a7, user_id=212d35a7e33046c59e3bace23a863d2a, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:09.545821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcf091-a119-4443-9faf-2db11df17bdb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:09.556505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1293/2563] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:09 2022] PUT /identity/v3/projects/91cd2a8105844171a5f3ab694ab3f8a7/users/212d35a7e33046c59e3bace23a863d2a/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:09.567564 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86020f-d3e4-4f61-86cd-24c402d50046 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:09.567953 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86020f-d3e4-4f61-86cd-24c402d50046 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:09.568289 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86020f-d3e4-4f61-86cd-24c402d50046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:09.599825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-7b86020f-d3e4-4f61-86cd-24c402d50046 None None] MFA Rules not processed for user `212d35a7e33046c59e3bace23a863d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:09.601260 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7b86020f-d3e4-4f61-86cd-24c402d50046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:09.681901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1271/2564] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:09.694971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcf091-a119-4443-9faf-2db11df17bdb admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:09.696730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-1ffcf091-a119-4443-9faf-2db11df17bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:09.774807 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcf091-a119-4443-9faf-2db11df17bdb admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:09.783025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deca3d5-94b0-4e1e-90a7-82939bd62de6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:09.783953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deca3d5-94b0-4e1e-90a7-82939bd62de6 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:09.784241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deca3d5-94b0-4e1e-90a7-82939bd62de6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:09.784559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deca3d5-94b0-4e1e-90a7-82939bd62de6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:09.797205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deca3d5-94b0-4e1e-90a7-82939bd62de6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:09.800377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deca3d5-94b0-4e1e-90a7-82939bd62de6 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:09.815307 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1294/2565] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:09.935961 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86020f-d3e4-4f61-86cd-24c402d50046 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:09.948238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86020f-d3e4-4f61-86cd-24c402d50046 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:09.952510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411aa0d-2bee-41fe-92cb-2de59586104a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:09.953540 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411aa0d-2bee-41fe-92cb-2de59586104a service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:09.953714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411aa0d-2bee-41fe-92cb-2de59586104a service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:09.954185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411aa0d-2bee-41fe-92cb-2de59586104a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:10.023929 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411aa0d-2bee-41fe-92cb-2de59586104a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:10.025887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411aa0d-2bee-41fe-92cb-2de59586104a service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:10.038625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1272/2566] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:11.946370 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deca3d5-94b0-4e1e-90a7-82939bd62de6 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:11.948246 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4deca3d5-94b0-4e1e-90a7-82939bd62de6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:12.030780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deca3d5-94b0-4e1e-90a7-82939bd62de6 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:12.041221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:12.042203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:12.042644 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:12.043086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] PATH_INFO: `/v3/users/b96eca838aba40ddb2e5b3c4f52c883d` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:12.044471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b96eca838aba40ddb2e5b3c4f52c883d)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:12.046696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:12.071509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:12.115228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b96eca838aba40ddb2e5b3c4f52c883d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:12.122205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b96eca838aba40ddb2e5b3c4f52c883d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:12.122903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b96eca838aba40ddb2e5b3c4f52c883d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:12.140784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b96eca838aba40ddb2e5b3c4f52c883d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:12.149015 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b96eca838aba40ddb2e5b3c4f52c883d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:12.158050 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1295/2567] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:11 2022] DELETE /identity/v3/users/b96eca838aba40ddb2e5b3c4f52c883d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:12.159926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160002bd-d724-43bf-afcd-932ea98e0145 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:12.160932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160002bd-d724-43bf-afcd-932ea98e0145 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:12.161035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160002bd-d724-43bf-afcd-932ea98e0145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:12.180652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-160002bd-d724-43bf-afcd-932ea98e0145 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:12.181873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-160002bd-d724-43bf-afcd-932ea98e0145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:12.199966 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:12.240612 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1273/2568] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:12.250686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:12.252189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:12.319250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81b59f-897a-4a2c-99a5-7803cc8556fb admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:12.322835 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656c15d-e780-4b45-b6d9-126f0d999a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:12.323439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656c15d-e780-4b45-b6d9-126f0d999a27 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:12.323744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656c15d-e780-4b45-b6d9-126f0d999a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:12.324016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656c15d-e780-4b45-b6d9-126f0d999a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:12.333443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656c15d-e780-4b45-b6d9-126f0d999a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:12.335483 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656c15d-e780-4b45-b6d9-126f0d999a27 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:12.344818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1296/2569] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:12.656851 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160002bd-d724-43bf-afcd-932ea98e0145 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:12.658840 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-160002bd-d724-43bf-afcd-932ea98e0145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:12.672948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160002bd-d724-43bf-afcd-932ea98e0145 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:12.677764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:12.678431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:12.678726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:12.678998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] PATH_INFO: `/v3/projects/499fb4b52df7483cbc30dca81e5f0ce1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:12.680192 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] RBAC: Authorizing `identity:delete_project(project_id=499fb4b52df7483cbc30dca81e5f0ce1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:12.682105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:12.742543 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '499fb4b52df7483cbc30dca81e5f0ce1', 'request_id': 'req-76043444-df44-4150-b2c5-6f3f6c46f31d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:12.749270 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '499fb4b52df7483cbc30dca81e5f0ce1', 'request_id': 'req-76043444-df44-4150-b2c5-6f3f6c46f31d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:12.759844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '499fb4b52df7483cbc30dca81e5f0ce1', 'request_id': 'req-76043444-df44-4150-b2c5-6f3f6c46f31d'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:12.773011 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] The token cache is being invalidate because project 499fb4b52df7483cbc30dca81e5f0ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:12.773307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:12.774003 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:12.775588 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1274/2570] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:12 2022] DELETE /identity/v3/projects/499fb4b52df7483cbc30dca81e5f0ce1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:12.785002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3de2-631e-49b2-8e9c-21edeb6fa872 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:12.785002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3de2-631e-49b2-8e9c-21edeb6fa872 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:12.785002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3de2-631e-49b2-8e9c-21edeb6fa872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:12.786163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:12.787876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:12.806714 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-6adc3de2-631e-49b2-8e9c-21edeb6fa872 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:12.808107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6adc3de2-631e-49b2-8e9c-21edeb6fa872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:12.870859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76043444-df44-4150-b2c5-6f3f6c46f31d admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:12.876666 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1297/2571] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:12.882029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecbf4c-fba4-4001-bd45-bfb67e3cabff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:12.882936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecbf4c-fba4-4001-bd45-bfb67e3cabff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:12.883319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecbf4c-fba4-4001-bd45-bfb67e3cabff admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:12.883765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecbf4c-fba4-4001-bd45-bfb67e3cabff admin admin] PATH_INFO: `/v3/users/2680099c0a2249449da37557ae58880c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:12.885251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecbf4c-fba4-4001-bd45-bfb67e3cabff admin admin] RBAC: Authorizing `identity:delete_user(user_id=2680099c0a2249449da37557ae58880c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:12.885631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a9c2-f7b3-4842-a837-dfab0d2fec59 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:12.886282 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a9c2-f7b3-4842-a837-dfab0d2fec59 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:12.886559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a9c2-f7b3-4842-a837-dfab0d2fec59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:12.888081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecbf4c-fba4-4001-bd45-bfb67e3cabff admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:12.913213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3eb4a9c2-f7b3-4842-a837-dfab0d2fec59 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:12.914580 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3eb4a9c2-f7b3-4842-a837-dfab0d2fec59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:12.924249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1298/2572] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:12.928889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a9c2-f7b3-4842-a837-dfab0d2fec59 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:12.945224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a9c2-f7b3-4842-a837-dfab0d2fec59 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:12.954235 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2420f4-0a47-4af7-bccf-69176724a910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:12.955819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2420f4-0a47-4af7-bccf-69176724a910 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:12.956559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2420f4-0a47-4af7-bccf-69176724a910 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:12.957575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2420f4-0a47-4af7-bccf-69176724a910 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:12.958559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2420f4-0a47-4af7-bccf-69176724a910 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:12.959094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2420f4-0a47-4af7-bccf-69176724a910 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:12.964268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2420f4-0a47-4af7-bccf-69176724a910 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:12.979009 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1299/2573] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:59:12.983945 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2420f4-0a47-4af7-bccf-69176724a910 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:12.996939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2420f4-0a47-4af7-bccf-69176724a910 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:13.001119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bf2f2-08b9-485c-a566-285fad135f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:13.001872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bf2f2-08b9-485c-a566-285fad135f49 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:13.002221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bf2f2-08b9-485c-a566-285fad135f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:13.002624 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-49ecbf4c-fba4-4001-bd45-bfb67e3cabff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2680099c0a2249449da37557ae58880c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:13.002738 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bf2f2-08b9-485c-a566-285fad135f49 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:13.003334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bf2f2-08b9-485c-a566-285fad135f49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:13.005383 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bf2f2-08b9-485c-a566-285fad135f49 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:13.010671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:13.022402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-49ecbf4c-fba4-4001-bd45-bfb67e3cabff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2680099c0a2249449da37557ae58880c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:13.024761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-49ecbf4c-fba4-4001-bd45-bfb67e3cabff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2680099c0a2249449da37557ae58880c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:13.025675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-49ecbf4c-fba4-4001-bd45-bfb67e3cabff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2680099c0a2249449da37557ae58880c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:13.029334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:13.058360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-49ecbf4c-fba4-4001-bd45-bfb67e3cabff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2680099c0a2249449da37557ae58880c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:13.065024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1300/2574] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:12 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:13.075398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bf2f2-08b9-485c-a566-285fad135f49 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:13.078104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-404bf2f2-08b9-485c-a566-285fad135f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:13.091802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1275/2575] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:12 2022] DELETE /identity/v3/users/2680099c0a2249449da37557ae58880c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:13.096601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b7b15-ac32-4b51-95e9-f561c81a58d0 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:13.096998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b7b15-ac32-4b51-95e9-f561c81a58d0 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:13.097485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b7b15-ac32-4b51-95e9-f561c81a58d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:13.141572 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-259b7b15-ac32-4b51-95e9-f561c81a58d0 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:13.142089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-259b7b15-ac32-4b51-95e9-f561c81a58d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:13.182650 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bf2f2-08b9-485c-a566-285fad135f49 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:13.279172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287d012-a86e-4864-baee-a72a70eea600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:13.279939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287d012-a86e-4864-baee-a72a70eea600 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:13.283569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287d012-a86e-4864-baee-a72a70eea600 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:13.284036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287d012-a86e-4864-baee-a72a70eea600 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:13.284850 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287d012-a86e-4864-baee-a72a70eea600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:13.287827 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287d012-a86e-4864-baee-a72a70eea600 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:13.316068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1276/2576] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:13.325605 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b7b15-ac32-4b51-95e9-f561c81a58d0 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:13.329056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-259b7b15-ac32-4b51-95e9-f561c81a58d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:13.338086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1301/2577] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:13 2022] POST /identity/v3/users => generated 288 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:13.342350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287d012-a86e-4864-baee-a72a70eea600 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:13.354523 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287d012-a86e-4864-baee-a72a70eea600 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:13.361565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8a01f-bb32-4072-b748-14a32144c8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:13.362174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8a01f-bb32-4072-b748-14a32144c8ea admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:13.362452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8a01f-bb32-4072-b748-14a32144c8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:13.362722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8a01f-bb32-4072-b748-14a32144c8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:13.363939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8a01f-bb32-4072-b748-14a32144c8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:13.365768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8a01f-bb32-4072-b748-14a32144c8ea admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:13.378670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1302/2578] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:13.382750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8a01f-bb32-4072-b748-14a32144c8ea admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:13.394719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8a01f-bb32-4072-b748-14a32144c8ea admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:13.400086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d69d09-d2b7-4186-a790-94e270b2fdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:13.400382 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b7b15-ac32-4b51-95e9-f561c81a58d0 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:13.401054 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d69d09-d2b7-4186-a790-94e270b2fdbf admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:13.401337 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d69d09-d2b7-4186-a790-94e270b2fdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:13.401611 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d69d09-d2b7-4186-a790-94e270b2fdbf admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:13.402244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d69d09-d2b7-4186-a790-94e270b2fdbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:13.403897 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d69d09-d2b7-4186-a790-94e270b2fdbf admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:13.404819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97393908-62b2-49e8-9050-821cccf7d2c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:13.405497 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97393908-62b2-49e8-9050-821cccf7d2c4 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:13.405850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97393908-62b2-49e8-9050-821cccf7d2c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:13.406198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97393908-62b2-49e8-9050-821cccf7d2c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:13.418630 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1303/2579] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:13.418809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97393908-62b2-49e8-9050-821cccf7d2c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:13.418809 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97393908-62b2-49e8-9050-821cccf7d2c4 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:13.419794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d69d09-d2b7-4186-a790-94e270b2fdbf admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:13.430221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1277/2580] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:13.430606 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d69d09-d2b7-4186-a790-94e270b2fdbf admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:13.435408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22819260-4585-4db1-9abe-51b356ae767d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:13.436105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22819260-4585-4db1-9abe-51b356ae767d admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:13.436387 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22819260-4585-4db1-9abe-51b356ae767d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:13.436656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22819260-4585-4db1-9abe-51b356ae767d admin admin] PATH_INFO: `/v3/projects/76ecb6d53b5146bb98eebef2718373ec/users/b467ab02084b482a8ce39471e4381dfd/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:13.453822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22819260-4585-4db1-9abe-51b356ae767d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ecb6d53b5146bb98eebef2718373ec, user_id=b467ab02084b482a8ce39471e4381dfd, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:13.456635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22819260-4585-4db1-9abe-51b356ae767d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:13.472556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1304/2581] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:13 2022] PUT /identity/v3/projects/76ecb6d53b5146bb98eebef2718373ec/users/b467ab02084b482a8ce39471e4381dfd/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:13.545425 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98bb36-5b45-4799-9dde-db76ebbbb314 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:13.545714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98bb36-5b45-4799-9dde-db76ebbbb314 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:13.546098 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98bb36-5b45-4799-9dde-db76ebbbb314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:13.576921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-ab98bb36-5b45-4799-9dde-db76ebbbb314 None None] MFA Rules not processed for user `b467ab02084b482a8ce39471e4381dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:13.578160 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ab98bb36-5b45-4799-9dde-db76ebbbb314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:13.680636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:13.685143 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1278/2582] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:13.697571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22819260-4585-4db1-9abe-51b356ae767d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:13.699403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-22819260-4585-4db1-9abe-51b356ae767d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:13.760218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98bb36-5b45-4799-9dde-db76ebbbb314 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:13.762794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ab98bb36-5b45-4799-9dde-db76ebbbb314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:13.789415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:13.812180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22819260-4585-4db1-9abe-51b356ae767d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:13.819359 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264f7d8-77c0-4614-a1ac-e4f9ea223263 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:13.820500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264f7d8-77c0-4614-a1ac-e4f9ea223263 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:13.820810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264f7d8-77c0-4614-a1ac-e4f9ea223263 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:13.821082 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264f7d8-77c0-4614-a1ac-e4f9ea223263 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:13.831561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264f7d8-77c0-4614-a1ac-e4f9ea223263 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:13.832707 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264f7d8-77c0-4614-a1ac-e4f9ea223263 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:13.843263 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1305/2583] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:13.878475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98bb36-5b45-4799-9dde-db76ebbbb314 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:13.884413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:13.885128 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:13.885484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:13.885829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9 admin admin] PATH_INFO: `/v3/projects/03954171dedd434f882c7d1a48cbb0b8` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:13.887066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03954171dedd434f882c7d1a48cbb0b8)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:13.889357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:13.957562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03954171dedd434f882c7d1a48cbb0b8', 'request_id': 'req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:13.965059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03954171dedd434f882c7d1a48cbb0b8', 'request_id': 'req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:13.974510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03954171dedd434f882c7d1a48cbb0b8', 'request_id': 'req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:13.991512 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9 admin admin] The token cache is being invalidate because project 03954171dedd434f882c7d1a48cbb0b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:13.991976 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:13.992738 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ccc2b02e-6278-4ad1-9671-abfc8e73dbd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:14.006165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1279/2584] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:13 2022] DELETE /identity/v3/projects/03954171dedd434f882c7d1a48cbb0b8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:14.006255 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264f7d8-77c0-4614-a1ac-e4f9ea223263 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:14.012341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264f7d8-77c0-4614-a1ac-e4f9ea223263 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:14.079334 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:14.080055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:14.080443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:14.080813 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] PATH_INFO: `/v3/users/212d35a7e33046c59e3bace23a863d2a` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:14.082796 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] RBAC: Authorizing `identity:delete_user(user_id=212d35a7e33046c59e3bace23a863d2a)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:14.085132 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:14.139218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212d35a7e33046c59e3bace23a863d2a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:14.148352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212d35a7e33046c59e3bace23a863d2a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:14.149188 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212d35a7e33046c59e3bace23a863d2a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:14.162775 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212d35a7e33046c59e3bace23a863d2a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:14.171927 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212d35a7e33046c59e3bace23a863d2a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:14.180886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1306/2585] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:13 2022] DELETE /identity/v3/users/212d35a7e33046c59e3bace23a863d2a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:14.479476 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512ebfc-c796-4bf1-a161-30666f88da98 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:14.480005 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512ebfc-c796-4bf1-a161-30666f88da98 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:14.480142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512ebfc-c796-4bf1-a161-30666f88da98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:14.493569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:14.495236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:14.502633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-c512ebfc-c796-4bf1-a161-30666f88da98 None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:14.503979 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c512ebfc-c796-4bf1-a161-30666f88da98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:14.567657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80f832-cfe9-4a33-b09b-7a0c9256e46d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:14.570832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1280/2586] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:14.572817 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8937b-ce9d-41b8-81ad-8b54e144813d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:14.573447 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8937b-ce9d-41b8-81ad-8b54e144813d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:14.573729 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8937b-ce9d-41b8-81ad-8b54e144813d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:14.574003 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8937b-ce9d-41b8-81ad-8b54e144813d admin admin] PATH_INFO: `/v3/projects/91cd2a8105844171a5f3ab694ab3f8a7` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:14.575134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8937b-ce9d-41b8-81ad-8b54e144813d admin admin] RBAC: Authorizing `identity:delete_project(project_id=91cd2a8105844171a5f3ab694ab3f8a7)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:14.577249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8937b-ce9d-41b8-81ad-8b54e144813d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:14.585942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:14.594268 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512ebfc-c796-4bf1-a161-30666f88da98 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:14.595671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c512ebfc-c796-4bf1-a161-30666f88da98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:14.631857 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ade8937b-ce9d-41b8-81ad-8b54e144813d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91cd2a8105844171a5f3ab694ab3f8a7', 'request_id': 'req-ade8937b-ce9d-41b8-81ad-8b54e144813d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:14.635979 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:14.637587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ade8937b-ce9d-41b8-81ad-8b54e144813d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91cd2a8105844171a5f3ab694ab3f8a7', 'request_id': 'req-ade8937b-ce9d-41b8-81ad-8b54e144813d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:14.648072 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ade8937b-ce9d-41b8-81ad-8b54e144813d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91cd2a8105844171a5f3ab694ab3f8a7', 'request_id': 'req-ade8937b-ce9d-41b8-81ad-8b54e144813d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:14.660731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ade8937b-ce9d-41b8-81ad-8b54e144813d admin admin] The token cache is being invalidate because project 91cd2a8105844171a5f3ab694ab3f8a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:14.660984 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ade8937b-ce9d-41b8-81ad-8b54e144813d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:14.661850 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-ade8937b-ce9d-41b8-81ad-8b54e144813d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:14.663848 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1307/2587] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:14 2022] DELETE /identity/v3/projects/91cd2a8105844171a5f3ab694ab3f8a7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:14.668604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827cf02-08a7-4e46-8622-4bb06514fc0b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:14.668935 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827cf02-08a7-4e46-8622-4bb06514fc0b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:14.669288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827cf02-08a7-4e46-8622-4bb06514fc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:14.674637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512ebfc-c796-4bf1-a161-30666f88da98 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:14.695794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8827cf02-08a7-4e46-8622-4bb06514fc0b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:14.697011 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8827cf02-08a7-4e46-8622-4bb06514fc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:14.736984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a8c41-00aa-4fb3-97b7-185d7358962c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:14.737601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a8c41-00aa-4fb3-97b7-185d7358962c service swift] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:14.737880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a8c41-00aa-4fb3-97b7-185d7358962c service swift] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:14.738158 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a8c41-00aa-4fb3-97b7-185d7358962c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:14.740593 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-313a8c41-00aa-4fb3-97b7-185d7358962c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:14.760476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1308/2588] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:14.768881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24efdc-80a0-4611-b38d-09972d377b2b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:14.769195 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24efdc-80a0-4611-b38d-09972d377b2b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:14.769510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24efdc-80a0-4611-b38d-09972d377b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:14.789336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3e24efdc-80a0-4611-b38d-09972d377b2b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:14.790555 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3e24efdc-80a0-4611-b38d-09972d377b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:14.798442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1309/2589] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:14.801782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a8c41-00aa-4fb3-97b7-185d7358962c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:14.803333 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24efdc-80a0-4611-b38d-09972d377b2b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:14.803490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a8c41-00aa-4fb3-97b7-185d7358962c service swift] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:14.811815 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1281/2590] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:59:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 18 14:59:14.812302 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24efdc-80a0-4611-b38d-09972d377b2b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:14.817313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80000c4b-5d32-49b9-88ea-3696c876fe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:14.817921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80000c4b-5d32-49b9-88ea-3696c876fe2d admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:14.818187 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80000c4b-5d32-49b9-88ea-3696c876fe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:14.818467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80000c4b-5d32-49b9-88ea-3696c876fe2d admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:14.819114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80000c4b-5d32-49b9-88ea-3696c876fe2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:14.819423 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80000c4b-5d32-49b9-88ea-3696c876fe2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:14.821171 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80000c4b-5d32-49b9-88ea-3696c876fe2d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:14.834347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1310/2591] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:59:14.838723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a8c41-00aa-4fb3-97b7-185d7358962c service swift] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:14.852059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a8c41-00aa-4fb3-97b7-185d7358962c service swift] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:14.858036 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6cec6-028e-405b-8be9-2d6574d3b9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:14.858666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6cec6-028e-405b-8be9-2d6574d3b9ab admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:14.858943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6cec6-028e-405b-8be9-2d6574d3b9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:14.859218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6cec6-028e-405b-8be9-2d6574d3b9ab admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:14.859895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6cec6-028e-405b-8be9-2d6574d3b9ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:14.862250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6cec6-028e-405b-8be9-2d6574d3b9ab admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:14.896795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1282/2592] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:14 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:59:14.898061 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80000c4b-5d32-49b9-88ea-3696c876fe2d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:14.908756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80000c4b-5d32-49b9-88ea-3696c876fe2d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:14.975176 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee75be3-47fb-43b3-921e-504f4c8ea4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:14.975811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee75be3-47fb-43b3-921e-504f4c8ea4fb admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:14.976088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee75be3-47fb-43b3-921e-504f4c8ea4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:14.976358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee75be3-47fb-43b3-921e-504f4c8ea4fb admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:14.976971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee75be3-47fb-43b3-921e-504f4c8ea4fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:14.978900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee75be3-47fb-43b3-921e-504f4c8ea4fb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:15.002920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1311/2593] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:14 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:15.004264 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6cec6-028e-405b-8be9-2d6574d3b9ab admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:15.015934 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6cec6-028e-405b-8be9-2d6574d3b9ab admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:15.021860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697849ee-29f8-489e-910a-48e3dee2f6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:15.022498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697849ee-29f8-489e-910a-48e3dee2f6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.022781 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697849ee-29f8-489e-910a-48e3dee2f6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.023059 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697849ee-29f8-489e-910a-48e3dee2f6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.023723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697849ee-29f8-489e-910a-48e3dee2f6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:15.025357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697849ee-29f8-489e-910a-48e3dee2f6a2 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:15.037919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1283/2594] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:15.042198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee75be3-47fb-43b3-921e-504f4c8ea4fb admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:15.049982 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:15.055416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee75be3-47fb-43b3-921e-504f4c8ea4fb admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:15.063773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a021f-bb52-48f5-9fc4-ac927d75c87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:15.064384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a021f-bb52-48f5-9fc4-ac927d75c87b admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.064676 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a021f-bb52-48f5-9fc4-ac927d75c87b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.065012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a021f-bb52-48f5-9fc4-ac927d75c87b admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.065947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a021f-bb52-48f5-9fc4-ac927d75c87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:15.067805 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a021f-bb52-48f5-9fc4-ac927d75c87b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:15.089942 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1312/2595] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:15.094083 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697849ee-29f8-489e-910a-48e3dee2f6a2 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:15.109098 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697849ee-29f8-489e-910a-48e3dee2f6a2 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:15.114396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248eea0-d4a0-4179-b837-d363953758e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:15.115257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248eea0-d4a0-4179-b837-d363953758e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.115617 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248eea0-d4a0-4179-b837-d363953758e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.116061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248eea0-d4a0-4179-b837-d363953758e4 admin admin] PATH_INFO: `/v3/projects/900a4c548fab4d628e7629a90b0dfea1/users/6c858c63569243549266214ce1d95fd2/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.137607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248eea0-d4a0-4179-b837-d363953758e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=900a4c548fab4d628e7629a90b0dfea1, user_id=6c858c63569243549266214ce1d95fd2, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:15.140580 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248eea0-d4a0-4179-b837-d363953758e4 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:15.159377 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1284/2596] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:15 2022] PUT /identity/v3/projects/900a4c548fab4d628e7629a90b0dfea1/users/6c858c63569243549266214ce1d95fd2/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:15.174545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0699b-82ce-4330-98bb-cccb459fe6bc None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.174904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0699b-82ce-4330-98bb-cccb459fe6bc None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.175101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0699b-82ce-4330-98bb-cccb459fe6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.209278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-69b0699b-82ce-4330-98bb-cccb459fe6bc None None] MFA Rules not processed for user `6c858c63569243549266214ce1d95fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:15.211461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-69b0699b-82ce-4330-98bb-cccb459fe6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:15.285402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1313/2597] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:15.289645 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248eea0-d4a0-4179-b837-d363953758e4 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:15.320433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:15.368465 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248eea0-d4a0-4179-b837-d363953758e4 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:15.375351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b6f9f-eb11-4d0e-b3e8-15c31e88369c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:15.376176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b6f9f-eb11-4d0e-b3e8-15c31e88369c admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.376176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b6f9f-eb11-4d0e-b3e8-15c31e88369c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.376620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b6f9f-eb11-4d0e-b3e8-15c31e88369c admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.377423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b6f9f-eb11-4d0e-b3e8-15c31e88369c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:15.379995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b6f9f-eb11-4d0e-b3e8-15c31e88369c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:15.420732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1285/2598] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:15 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:59:15.422452 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0699b-82ce-4330-98bb-cccb459fe6bc None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:15.432579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9045a2-5df8-4f88-9b82-a533316b449a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.433029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9045a2-5df8-4f88-9b82-a533316b449a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.433369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9045a2-5df8-4f88-9b82-a533316b449a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.438340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0699b-82ce-4330-98bb-cccb459fe6bc None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:15.453704 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-ed9045a2-5df8-4f88-9b82-a533316b449a None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:15.454982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ed9045a2-5df8-4f88-9b82-a533316b449a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:15.510969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78ceeb-9e0c-4463-b4cc-b341af4860b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:15.512491 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78ceeb-9e0c-4463-b4cc-b341af4860b4 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.512809 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78ceeb-9e0c-4463-b4cc-b341af4860b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.513099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78ceeb-9e0c-4463-b4cc-b341af4860b4 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.514310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78ceeb-9e0c-4463-b4cc-b341af4860b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:15.516741 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78ceeb-9e0c-4463-b4cc-b341af4860b4 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:15.526766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1286/2599] 10.222.0.36 () {58 vars in 967 bytes} [Thu Aug 18 14:59:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:15.543170 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1314/2600] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:15 2022] POST /identity/v3/users => generated 301 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:15.546172 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9045a2-5df8-4f88-9b82-a533316b449a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:15.546965 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78ceeb-9e0c-4463-b4cc-b341af4860b4 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:15.547801 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ed9045a2-5df8-4f88-9b82-a533316b449a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:15.558682 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78ceeb-9e0c-4463-b4cc-b341af4860b4 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:15.563530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc463f4d-c865-4014-9b21-97fb602121e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:15.564892 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc463f4d-c865-4014-9b21-97fb602121e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.564892 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc463f4d-c865-4014-9b21-97fb602121e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.564892 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc463f4d-c865-4014-9b21-97fb602121e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.565435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc463f4d-c865-4014-9b21-97fb602121e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:15.567261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc463f4d-c865-4014-9b21-97fb602121e8 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:15.581358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1315/2601] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:15.585516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc463f4d-c865-4014-9b21-97fb602121e8 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:15.599789 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc463f4d-c865-4014-9b21-97fb602121e8 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:15.606021 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aac10-5261-4cf6-91ca-e45a93b947dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:15.606771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aac10-5261-4cf6-91ca-e45a93b947dd admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.607052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aac10-5261-4cf6-91ca-e45a93b947dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.607405 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aac10-5261-4cf6-91ca-e45a93b947dd admin admin] PATH_INFO: `/v3/projects/a978551e4e4845c6aa06cf576295348b/users/eef568d698f64146bbbffa0792894210/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.632345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613aac10-5261-4cf6-91ca-e45a93b947dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a978551e4e4845c6aa06cf576295348b, user_id=eef568d698f64146bbbffa0792894210, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:15.635396 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613aac10-5261-4cf6-91ca-e45a93b947dd admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:15.645171 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9045a2-5df8-4f88-9b82-a533316b449a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:15.650977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa801031-a4e7-4a06-b41d-4e63f32d7fe6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:15.653218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa801031-a4e7-4a06-b41d-4e63f32d7fe6 service swift] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.653758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa801031-a4e7-4a06-b41d-4e63f32d7fe6 service swift] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.656063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa801031-a4e7-4a06-b41d-4e63f32d7fe6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.660862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1316/2602] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:15 2022] PUT /identity/v3/projects/a978551e4e4845c6aa06cf576295348b/users/eef568d698f64146bbbffa0792894210/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:15.664704 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aac10-5261-4cf6-91ca-e45a93b947dd admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:15.671427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa801031-a4e7-4a06-b41d-4e63f32d7fe6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:15.673955 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa801031-a4e7-4a06-b41d-4e63f32d7fe6 service swift] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:15.683644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1287/2603] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:59:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:59:15.740341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:15.769967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aac10-5261-4cf6-91ca-e45a93b947dd admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:15.774865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98762c-3879-4ea3-a7e6-30bc3975b293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:15.775492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98762c-3879-4ea3-a7e6-30bc3975b293 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.775801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98762c-3879-4ea3-a7e6-30bc3975b293 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.776072 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98762c-3879-4ea3-a7e6-30bc3975b293 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.776755 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98762c-3879-4ea3-a7e6-30bc3975b293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:15.778506 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98762c-3879-4ea3-a7e6-30bc3975b293 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:15.791714 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1317/2604] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:15.794291 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa801031-a4e7-4a06-b41d-4e63f32d7fe6 service swift] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:15.804680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa801031-a4e7-4a06-b41d-4e63f32d7fe6 service swift] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:15.809253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08e786-55ce-4097-a4e6-f50a4b949e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:15.809900 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08e786-55ce-4097-a4e6-f50a4b949e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.810179 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08e786-55ce-4097-a4e6-f50a4b949e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.810451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08e786-55ce-4097-a4e6-f50a4b949e03 admin admin] PATH_INFO: `/v3/projects/a978551e4e4845c6aa06cf576295348b/users/eef568d698f64146bbbffa0792894210/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.812417 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08e786-55ce-4097-a4e6-f50a4b949e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a978551e4e4845c6aa06cf576295348b, user_id=eef568d698f64146bbbffa0792894210, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:15.815176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08e786-55ce-4097-a4e6-f50a4b949e03 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:15.825337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1288/2605] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:15 2022] PUT /identity/v3/projects/a978551e4e4845c6aa06cf576295348b/users/eef568d698f64146bbbffa0792894210/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:15.836287 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d8b4e-3b85-410f-bbf5-e1fb2c031057 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.836574 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d8b4e-3b85-410f-bbf5-e1fb2c031057 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.836851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d8b4e-3b85-410f-bbf5-e1fb2c031057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.866647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-d55d8b4e-3b85-410f-bbf5-e1fb2c031057 None None] MFA Rules not processed for user `eef568d698f64146bbbffa0792894210`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:15.867907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d55d8b4e-3b85-410f-bbf5-e1fb2c031057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:15.929853 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce9544-24aa-4d1d-8eb4-46b44349316d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:15.930144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce9544-24aa-4d1d-8eb4-46b44349316d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:15.930423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce9544-24aa-4d1d-8eb4-46b44349316d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:15.943351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1318/2606] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:15 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:15.950238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-4cce9544-24aa-4d1d-8eb4-46b44349316d None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:15.951591 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4cce9544-24aa-4d1d-8eb4-46b44349316d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:15.954464 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d8b4e-3b85-410f-bbf5-e1fb2c031057 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:15.955892 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d55d8b4e-3b85-410f-bbf5-e1fb2c031057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:15.978182 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:16.009765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1289/2607] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 14:59:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:16.016480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d8b4e-3b85-410f-bbf5-e1fb2c031057 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:16.020806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b53c0-1f83-4908-8127-4fcd87a889dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:16.021415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b53c0-1f83-4908-8127-4fcd87a889dd service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:16.021693 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b53c0-1f83-4908-8127-4fcd87a889dd service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:16.021966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b53c0-1f83-4908-8127-4fcd87a889dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:16.022858 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce9544-24aa-4d1d-8eb4-46b44349316d None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:16.031296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b53c0-1f83-4908-8127-4fcd87a889dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:16.032747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b53c0-1f83-4908-8127-4fcd87a889dd service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:16.042964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1319/2608] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:15 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:59:16.098873 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce9544-24aa-4d1d-8eb4-46b44349316d None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:16.106487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f75246-9b3e-45be-8e34-0a4a66c7ae10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:16.106487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f75246-9b3e-45be-8e34-0a4a66c7ae10 service swift] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:16.106487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f75246-9b3e-45be-8e34-0a4a66c7ae10 service swift] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:16.106487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f75246-9b3e-45be-8e34-0a4a66c7ae10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:16.115537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f75246-9b3e-45be-8e34-0a4a66c7ae10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:16.117203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f75246-9b3e-45be-8e34-0a4a66c7ae10 service swift] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:16.125571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1290/2609] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:59:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 18 14:59:16.185463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b53c0-1f83-4908-8127-4fcd87a889dd service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:16.251589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b53c0-1f83-4908-8127-4fcd87a889dd service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:16.258147 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b2e5b-d263-4866-a66f-248870679daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:16.258825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b2e5b-d263-4866-a66f-248870679daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:16.259101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b2e5b-d263-4866-a66f-248870679daf admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:16.259434 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b2e5b-d263-4866-a66f-248870679daf admin admin] PATH_INFO: `/v3/users/6c858c63569243549266214ce1d95fd2` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:16.260732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b2e5b-d263-4866-a66f-248870679daf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c858c63569243549266214ce1d95fd2)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:16.262631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b2e5b-d263-4866-a66f-248870679daf admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:16.265757 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:16.339219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-175b2e5b-d263-4866-a66f-248870679daf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c858c63569243549266214ce1d95fd2'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:16.348534 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-175b2e5b-d263-4866-a66f-248870679daf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c858c63569243549266214ce1d95fd2'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:16.349230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-175b2e5b-d263-4866-a66f-248870679daf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c858c63569243549266214ce1d95fd2'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:16.363151 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-175b2e5b-d263-4866-a66f-248870679daf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c858c63569243549266214ce1d95fd2'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:16.379990 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-175b2e5b-d263-4866-a66f-248870679daf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c858c63569243549266214ce1d95fd2'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:16.384345 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f75246-9b3e-45be-8e34-0a4a66c7ae10 service swift] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:16.385765 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-85f75246-9b3e-45be-8e34-0a4a66c7ae10 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:16.387367 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1320/2610] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:16 2022] DELETE /identity/v3/users/6c858c63569243549266214ce1d95fd2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:16.392034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76749129-04da-4d09-8813-4c268503a892 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:16.392322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76749129-04da-4d09-8813-4c268503a892 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:16.392598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76749129-04da-4d09-8813-4c268503a892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:16.413238 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-76749129-04da-4d09-8813-4c268503a892 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:16.414561 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-76749129-04da-4d09-8813-4c268503a892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:16.452998 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f75246-9b3e-45be-8e34-0a4a66c7ae10 service swift] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:16.493119 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1321/2611] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:16.494372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6e9f6-8859-4454-92df-ff5bb7d4b1ca None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:16.494654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6e9f6-8859-4454-92df-ff5bb7d4b1ca None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:16.494923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6e9f6-8859-4454-92df-ff5bb7d4b1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:16.517514 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-c5d6e9f6-8859-4454-92df-ff5bb7d4b1ca None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:16.518486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a6530-11b6-4f47-bfd2-425dfb59d825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:16.518763 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c5d6e9f6-8859-4454-92df-ff5bb7d4b1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:16.519187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a6530-11b6-4f47-bfd2-425dfb59d825 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:16.520035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a6530-11b6-4f47-bfd2-425dfb59d825 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:16.520314 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a6530-11b6-4f47-bfd2-425dfb59d825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:16.522107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-eb3a6530-11b6-4f47-bfd2-425dfb59d825 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:16.535925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a6530-11b6-4f47-bfd2-425dfb59d825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:16.537369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a6530-11b6-4f47-bfd2-425dfb59d825 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:16.547344 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1291/2612] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:59:16.548318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a6530-11b6-4f47-bfd2-425dfb59d825 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:16.549594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-eb3a6530-11b6-4f47-bfd2-425dfb59d825 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:16.561643 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a6530-11b6-4f47-bfd2-425dfb59d825 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:16.565727 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594e385-8ff5-415f-94ea-6a72a104962d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:16.566357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e385-8ff5-415f-94ea-6a72a104962d service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:16.566641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e385-8ff5-415f-94ea-6a72a104962d service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:16.566916 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e385-8ff5-415f-94ea-6a72a104962d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:16.576618 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594e385-8ff5-415f-94ea-6a72a104962d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:16.578061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594e385-8ff5-415f-94ea-6a72a104962d service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:16.583236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1322/2613] 10.222.0.36 () {58 vars in 967 bytes} [Thu Aug 18 14:59:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:16.588619 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1292/2614] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:16.597007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6e9f6-8859-4454-92df-ff5bb7d4b1ca None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:16.599317 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c5d6e9f6-8859-4454-92df-ff5bb7d4b1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:16.631203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:16.684872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6e9f6-8859-4454-92df-ff5bb7d4b1ca None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:16.690515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f8f37-3ff5-4b3f-98ed-89138d276c0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:16.691124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f8f37-3ff5-4b3f-98ed-89138d276c0e service swift] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:16.691463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f8f37-3ff5-4b3f-98ed-89138d276c0e service swift] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:16.692328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f8f37-3ff5-4b3f-98ed-89138d276c0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:16.701963 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f8f37-3ff5-4b3f-98ed-89138d276c0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:16.704596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f8f37-3ff5-4b3f-98ed-89138d276c0e service swift] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:16.719548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1323/2615] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 14:59:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 14:59:17.131549 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594e385-8ff5-415f-94ea-6a72a104962d service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:17.133156 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8594e385-8ff5-415f-94ea-6a72a104962d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:17.156723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594e385-8ff5-415f-94ea-6a72a104962d service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:17.184360 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:17.185015 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:17.185352 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:17.185583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] PATH_INFO: `/v3/projects/900a4c548fab4d628e7629a90b0dfea1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:17.193436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] RBAC: Authorizing `identity:delete_project(project_id=900a4c548fab4d628e7629a90b0dfea1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:17.196868 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:17.200187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:17.271901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '900a4c548fab4d628e7629a90b0dfea1', 'request_id': 'req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:17.287376 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '900a4c548fab4d628e7629a90b0dfea1', 'request_id': 'req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:17.309407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '900a4c548fab4d628e7629a90b0dfea1', 'request_id': 'req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:17.334778 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] The token cache is being invalidate because project 900a4c548fab4d628e7629a90b0dfea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:17.335077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:17.335831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:17.338081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1293/2616] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:17 2022] DELETE /identity/v3/projects/900a4c548fab4d628e7629a90b0dfea1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:17.342686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f8f37-3ff5-4b3f-98ed-89138d276c0e service swift] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:17.355554 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f8f37-3ff5-4b3f-98ed-89138d276c0e service swift] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:17.434512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354e836-df22-426e-8f0f-83e67dee8378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:17.435627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354e836-df22-426e-8f0f-83e67dee8378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:17.436142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354e836-df22-426e-8f0f-83e67dee8378 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:17.436581 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354e836-df22-426e-8f0f-83e67dee8378 admin admin] PATH_INFO: `/v3/users/eef568d698f64146bbbffa0792894210` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:17.437910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354e836-df22-426e-8f0f-83e67dee8378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef568d698f64146bbbffa0792894210)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:17.440045 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354e836-df22-426e-8f0f-83e67dee8378 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:17.500744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8354e836-df22-426e-8f0f-83e67dee8378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef568d698f64146bbbffa0792894210'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:17.509521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8354e836-df22-426e-8f0f-83e67dee8378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef568d698f64146bbbffa0792894210'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:17.510324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8354e836-df22-426e-8f0f-83e67dee8378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef568d698f64146bbbffa0792894210'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:17.525850 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8354e836-df22-426e-8f0f-83e67dee8378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef568d698f64146bbbffa0792894210'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:17.534889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-8354e836-df22-426e-8f0f-83e67dee8378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef568d698f64146bbbffa0792894210'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:17.547766 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1324/2617] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:17 2022] DELETE /identity/v3/users/eef568d698f64146bbbffa0792894210 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:17.903284 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:17.905424 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:17.938185 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd4cb0-fc56-4f3e-9e99-1290b843f59a admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:18.033959 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742ccec-a8e9-4932-8e98-65545b15a1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:18.034612 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742ccec-a8e9-4932-8e98-65545b15a1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:18.035024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742ccec-a8e9-4932-8e98-65545b15a1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:18.035301 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742ccec-a8e9-4932-8e98-65545b15a1b1 admin admin] PATH_INFO: `/v3/projects/a978551e4e4845c6aa06cf576295348b` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:18.036539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742ccec-a8e9-4932-8e98-65545b15a1b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a978551e4e4845c6aa06cf576295348b)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:18.039191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742ccec-a8e9-4932-8e98-65545b15a1b1 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:18.087284 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1742ccec-a8e9-4932-8e98-65545b15a1b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a978551e4e4845c6aa06cf576295348b', 'request_id': 'req-1742ccec-a8e9-4932-8e98-65545b15a1b1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:18.092789 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1742ccec-a8e9-4932-8e98-65545b15a1b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a978551e4e4845c6aa06cf576295348b', 'request_id': 'req-1742ccec-a8e9-4932-8e98-65545b15a1b1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:18.100973 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1742ccec-a8e9-4932-8e98-65545b15a1b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a978551e4e4845c6aa06cf576295348b', 'request_id': 'req-1742ccec-a8e9-4932-8e98-65545b15a1b1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:18.112359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1742ccec-a8e9-4932-8e98-65545b15a1b1 admin admin] The token cache is being invalidate because project a978551e4e4845c6aa06cf576295348b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:18.112649 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1742ccec-a8e9-4932-8e98-65545b15a1b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:18.113832 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-1742ccec-a8e9-4932-8e98-65545b15a1b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:18.115584 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1294/2618] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:17 2022] DELETE /identity/v3/projects/a978551e4e4845c6aa06cf576295348b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:18.120378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df200e-af39-47c8-87c0-d99d81710b38 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:18.120736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df200e-af39-47c8-87c0-d99d81710b38 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:18.121098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df200e-af39-47c8-87c0-d99d81710b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:18.142386 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-99df200e-af39-47c8-87c0-d99d81710b38 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:18.144053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-99df200e-af39-47c8-87c0-d99d81710b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:18.177160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:18.216128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1325/2619] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:18.218576 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ad883-4c3b-4ace-9c44-436a6306fe41 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:18.218934 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ad883-4c3b-4ace-9c44-436a6306fe41 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:18.219209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ad883-4c3b-4ace-9c44-436a6306fe41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:18.240221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-022ad883-4c3b-4ace-9c44-436a6306fe41 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:18.241447 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-022ad883-4c3b-4ace-9c44-436a6306fe41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:18.255545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df200e-af39-47c8-87c0-d99d81710b38 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:18.261933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1295/2620] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:18.289600 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df200e-af39-47c8-87c0-d99d81710b38 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:18.297071 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046ada1-f760-46b7-b2bc-31c222707e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:18.298128 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046ada1-f760-46b7-b2bc-31c222707e61 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:18.298559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046ada1-f760-46b7-b2bc-31c222707e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:18.299116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046ada1-f760-46b7-b2bc-31c222707e61 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:18.309510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046ada1-f760-46b7-b2bc-31c222707e61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:18.309829 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046ada1-f760-46b7-b2bc-31c222707e61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:18.315567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046ada1-f760-46b7-b2bc-31c222707e61 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:18.346682 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1326/2621] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:59:18.347932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ad883-4c3b-4ace-9c44-436a6306fe41 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:18.357200 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ad883-4c3b-4ace-9c44-436a6306fe41 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:18.368073 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a9661-144e-4ef9-9e3d-5a3ea69d114f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:18.368073 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a9661-144e-4ef9-9e3d-5a3ea69d114f admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:18.368073 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a9661-144e-4ef9-9e3d-5a3ea69d114f admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:18.368073 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a9661-144e-4ef9-9e3d-5a3ea69d114f admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:18.368460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a9661-144e-4ef9-9e3d-5a3ea69d114f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:18.368460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a9661-144e-4ef9-9e3d-5a3ea69d114f admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:18.398760 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:18.421215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1296/2622] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:18 2022] POST /identity/v3/projects => generated 365 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:18.426521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046ada1-f760-46b7-b2bc-31c222707e61 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:18.438562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046ada1-f760-46b7-b2bc-31c222707e61 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:18.541143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332a22-bf47-4509-b39c-a55af390a509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:18.541864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332a22-bf47-4509-b39c-a55af390a509 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:18.542143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332a22-bf47-4509-b39c-a55af390a509 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:18.542475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332a22-bf47-4509-b39c-a55af390a509 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:18.543231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b332a22-bf47-4509-b39c-a55af390a509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:18.546794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b332a22-bf47-4509-b39c-a55af390a509 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:18.583221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1327/2623] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:18 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:18.587500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a9661-144e-4ef9-9e3d-5a3ea69d114f admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:18.599286 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a9661-144e-4ef9-9e3d-5a3ea69d114f admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:18.615706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f1f72-211a-4f81-847e-3d9811af2d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:18.615706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f1f72-211a-4f81-847e-3d9811af2d80 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:18.615706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f1f72-211a-4f81-847e-3d9811af2d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:18.615706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f1f72-211a-4f81-847e-3d9811af2d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:18.616157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f1f72-211a-4f81-847e-3d9811af2d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:18.616157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f1f72-211a-4f81-847e-3d9811af2d80 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:18.633744 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332a22-bf47-4509-b39c-a55af390a509 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:18.634483 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1297/2624] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:18.645449 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332a22-bf47-4509-b39c-a55af390a509 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:18.650349 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b69f37-1f34-4f53-b8cf-1315c862f9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:18.650967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b69f37-1f34-4f53-b8cf-1315c862f9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:18.651249 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b69f37-1f34-4f53-b8cf-1315c862f9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:18.651662 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b69f37-1f34-4f53-b8cf-1315c862f9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:18.652346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b69f37-1f34-4f53-b8cf-1315c862f9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:18.654156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b69f37-1f34-4f53-b8cf-1315c862f9b8 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:18.670366 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1328/2625] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:18.671958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f1f72-211a-4f81-847e-3d9811af2d80 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:18.683558 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f1f72-211a-4f81-847e-3d9811af2d80 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:18.690350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fc675-40e0-4662-b931-00d4da988662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:18.691116 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fc675-40e0-4662-b931-00d4da988662 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:18.692172 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fc675-40e0-4662-b931-00d4da988662 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:18.692661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fc675-40e0-4662-b931-00d4da988662 admin admin] PATH_INFO: `/v3/projects/d8e090229d414449a449b013410298be/users/9e36ea7f5f1542118bb9dc1efe78c42c/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:18.711366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256fc675-40e0-4662-b931-00d4da988662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e090229d414449a449b013410298be, user_id=9e36ea7f5f1542118bb9dc1efe78c42c, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:18.714232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256fc675-40e0-4662-b931-00d4da988662 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:18.740399 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1298/2626] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:18 2022] PUT /identity/v3/projects/d8e090229d414449a449b013410298be/users/9e36ea7f5f1542118bb9dc1efe78c42c/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:18.743799 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89b283-84d6-4f6c-8a22-3f4100fd796f None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:18.744092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89b283-84d6-4f6c-8a22-3f4100fd796f None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:18.744371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89b283-84d6-4f6c-8a22-3f4100fd796f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:18.778929 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8c89b283-84d6-4f6c-8a22-3f4100fd796f None None] MFA Rules not processed for user `9e36ea7f5f1542118bb9dc1efe78c42c`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:18.780193 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8c89b283-84d6-4f6c-8a22-3f4100fd796f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:18.878640 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1329/2627] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:18.884000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fc675-40e0-4662-b931-00d4da988662 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:18.970935 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fc675-40e0-4662-b931-00d4da988662 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:18.977636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b307b-f294-4bd0-9f1f-c8573490377c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:18.979271 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b307b-f294-4bd0-9f1f-c8573490377c admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:18.979271 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b307b-f294-4bd0-9f1f-c8573490377c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:18.979271 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b307b-f294-4bd0-9f1f-c8573490377c admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:18.979581 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b307b-f294-4bd0-9f1f-c8573490377c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:18.981941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b307b-f294-4bd0-9f1f-c8573490377c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:19.047235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1299/2628] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:18 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:19.055113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89b283-84d6-4f6c-8a22-3f4100fd796f None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:19.074957 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89b283-84d6-4f6c-8a22-3f4100fd796f None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:19.085909 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b307b-f294-4bd0-9f1f-c8573490377c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:19.088968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c16b307b-f294-4bd0-9f1f-c8573490377c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:19.112625 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:19.168582 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fdfdc9-3339-4f5c-a479-63ba04012dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:19.169406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdfdc9-3339-4f5c-a479-63ba04012dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:19.169518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdfdc9-3339-4f5c-a479-63ba04012dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:19.169736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdfdc9-3339-4f5c-a479-63ba04012dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:19.170535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fdfdc9-3339-4f5c-a479-63ba04012dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:19.173090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fdfdc9-3339-4f5c-a479-63ba04012dc9 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:19.192468 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b307b-f294-4bd0-9f1f-c8573490377c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:19.198039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d2cb2-8ac6-4360-95fa-83480c2c5c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:19.198912 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d2cb2-8ac6-4360-95fa-83480c2c5c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:19.199477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d2cb2-8ac6-4360-95fa-83480c2c5c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:19.200207 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d2cb2-8ac6-4360-95fa-83480c2c5c5e admin admin] PATH_INFO: `/v3/users/0bd506975b28441cae07cd3bceb04970` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:19.202521 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d2cb2-8ac6-4360-95fa-83480c2c5c5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bd506975b28441cae07cd3bceb04970)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:19.205316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d2cb2-8ac6-4360-95fa-83480c2c5c5e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:19.238338 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1330/2629] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:19 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:19.242265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fdfdc9-3339-4f5c-a479-63ba04012dc9 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:19.257804 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fdfdc9-3339-4f5c-a479-63ba04012dc9 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:19.267064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91748c7-8017-4b4d-b3e2-df383bb04553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:19.269194 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91748c7-8017-4b4d-b3e2-df383bb04553 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:19.269194 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91748c7-8017-4b4d-b3e2-df383bb04553 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:19.269460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91748c7-8017-4b4d-b3e2-df383bb04553 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:19.270872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91748c7-8017-4b4d-b3e2-df383bb04553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:19.274030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91748c7-8017-4b4d-b3e2-df383bb04553 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:19.288111 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1331/2630] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:19.292553 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91748c7-8017-4b4d-b3e2-df383bb04553 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:19.298887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:19.304852 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-296d2cb2-8ac6-4360-95fa-83480c2c5c5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bd506975b28441cae07cd3bceb04970'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:19.307479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91748c7-8017-4b4d-b3e2-df383bb04553 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:19.313568 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50057bf2-c9cd-48cb-8542-2cbf9d2e1ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:19.314475 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50057bf2-c9cd-48cb-8542-2cbf9d2e1ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:19.314755 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50057bf2-c9cd-48cb-8542-2cbf9d2e1ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:19.315352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50057bf2-c9cd-48cb-8542-2cbf9d2e1ed3 admin admin] PATH_INFO: `/v3/projects/43e3ed6833064b3e8cf38caa0e7d1cf2/users/c23aa2c853964a76832161d32ffa56ea/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:19.315644 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-296d2cb2-8ac6-4360-95fa-83480c2c5c5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bd506975b28441cae07cd3bceb04970'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:19.317329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-296d2cb2-8ac6-4360-95fa-83480c2c5c5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bd506975b28441cae07cd3bceb04970'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:19.318041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-296d2cb2-8ac6-4360-95fa-83480c2c5c5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bd506975b28441cae07cd3bceb04970'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:19.331418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-296d2cb2-8ac6-4360-95fa-83480c2c5c5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bd506975b28441cae07cd3bceb04970'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:19.337567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50057bf2-c9cd-48cb-8542-2cbf9d2e1ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43e3ed6833064b3e8cf38caa0e7d1cf2, user_id=c23aa2c853964a76832161d32ffa56ea, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:19.340767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50057bf2-c9cd-48cb-8542-2cbf9d2e1ed3 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:19.353557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1300/2631] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:19 2022] DELETE /identity/v3/users/0bd506975b28441cae07cd3bceb04970 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:19.357045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c96fc0-c403-4e39-b1da-83e2354685ea None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:19.357363 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c96fc0-c403-4e39-b1da-83e2354685ea None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:19.357643 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c96fc0-c403-4e39-b1da-83e2354685ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:19.368439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1332/2632] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:19 2022] PUT /identity/v3/projects/43e3ed6833064b3e8cf38caa0e7d1cf2/users/c23aa2c853964a76832161d32ffa56ea/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:19.369356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50057bf2-c9cd-48cb-8542-2cbf9d2e1ed3 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:19.372303 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-50057bf2-c9cd-48cb-8542-2cbf9d2e1ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:19.387117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a8c96fc0-c403-4e39-b1da-83e2354685ea None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:19.388596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a8c96fc0-c403-4e39-b1da-83e2354685ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:19.460213 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50057bf2-c9cd-48cb-8542-2cbf9d2e1ed3 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:19.472332 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb813a8-c16b-4aeb-8fd9-90591d612870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:19.472931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb813a8-c16b-4aeb-8fd9-90591d612870 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:19.473411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb813a8-c16b-4aeb-8fd9-90591d612870 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:19.473809 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb813a8-c16b-4aeb-8fd9-90591d612870 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:19.474439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb813a8-c16b-4aeb-8fd9-90591d612870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:19.476357 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb813a8-c16b-4aeb-8fd9-90591d612870 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:19.491506 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1333/2633] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:19.491855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1301/2634] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:19.498237 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb813a8-c16b-4aeb-8fd9-90591d612870 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:19.505475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c96fc0-c403-4e39-b1da-83e2354685ea None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:19.507030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a8c96fc0-c403-4e39-b1da-83e2354685ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:19.511344 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb813a8-c16b-4aeb-8fd9-90591d612870 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:19.516549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e672c-3aef-4d4b-82a4-aaf0fab79626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:19.517215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e672c-3aef-4d4b-82a4-aaf0fab79626 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:19.517769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e672c-3aef-4d4b-82a4-aaf0fab79626 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:19.518221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e672c-3aef-4d4b-82a4-aaf0fab79626 admin admin] PATH_INFO: `/v3/projects/43e3ed6833064b3e8cf38caa0e7d1cf2/users/c23aa2c853964a76832161d32ffa56ea/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:19.521786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e672c-3aef-4d4b-82a4-aaf0fab79626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43e3ed6833064b3e8cf38caa0e7d1cf2, user_id=c23aa2c853964a76832161d32ffa56ea, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:19.526569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e672c-3aef-4d4b-82a4-aaf0fab79626 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:19.544596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1334/2635] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:19 2022] PUT /identity/v3/projects/43e3ed6833064b3e8cf38caa0e7d1cf2/users/c23aa2c853964a76832161d32ffa56ea/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:19.555099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d2d41-26ce-488e-a0ee-0793f70f8de2 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:19.555384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d2d41-26ce-488e-a0ee-0793f70f8de2 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:19.555657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d2d41-26ce-488e-a0ee-0793f70f8de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:19.594204 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-dd6d2d41-26ce-488e-a0ee-0793f70f8de2 None None] MFA Rules not processed for user `c23aa2c853964a76832161d32ffa56ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:19.594204 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-dd6d2d41-26ce-488e-a0ee-0793f70f8de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:19.615498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c96fc0-c403-4e39-b1da-83e2354685ea None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:19.691382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:19.717269 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1335/2636] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:19.722241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e4818-aa1b-43a1-913d-ad755e6e742e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:19.723874 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4818-aa1b-43a1-913d-ad755e6e742e service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:19.724375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4818-aa1b-43a1-913d-ad755e6e742e service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:19.724834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4818-aa1b-43a1-913d-ad755e6e742e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:19.729743 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d2d41-26ce-488e-a0ee-0793f70f8de2 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:19.732555 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-dd6d2d41-26ce-488e-a0ee-0793f70f8de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:19.761759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:19.824926 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e4818-aa1b-43a1-913d-ad755e6e742e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:19.825367 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d2d41-26ce-488e-a0ee-0793f70f8de2 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:19.827105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e4818-aa1b-43a1-913d-ad755e6e742e service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:19.839131 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed73f1b-99fc-454d-947a-923cff1780f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:19.840340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed73f1b-99fc-454d-947a-923cff1780f5 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:19.841079 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed73f1b-99fc-454d-947a-923cff1780f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:19.841543 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed73f1b-99fc-454d-947a-923cff1780f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:19.848737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1302/2637] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:19.869727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed73f1b-99fc-454d-947a-923cff1780f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:19.874210 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed73f1b-99fc-454d-947a-923cff1780f5 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:19.887179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1336/2638] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:20.016202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e4818-aa1b-43a1-913d-ad755e6e742e service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:20.027505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e4818-aa1b-43a1-913d-ad755e6e742e service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:20.031488 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25b643-7b23-4184-87dc-443bffd662e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:20.033090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25b643-7b23-4184-87dc-443bffd662e0 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:20.033379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25b643-7b23-4184-87dc-443bffd662e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:20.033724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25b643-7b23-4184-87dc-443bffd662e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:20.035894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-db25b643-7b23-4184-87dc-443bffd662e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:20.148664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25b643-7b23-4184-87dc-443bffd662e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:20.150402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25b643-7b23-4184-87dc-443bffd662e0 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:20.179992 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1303/2639] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:20.696001 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed73f1b-99fc-454d-947a-923cff1780f5 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:20.697538 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9ed73f1b-99fc-454d-947a-923cff1780f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:20.715958 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed73f1b-99fc-454d-947a-923cff1780f5 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:20.729322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:20.730339 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:20.730913 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:20.731492 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] PATH_INFO: `/v3/projects/c12ac892aa6b4154b90e992217298624` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:20.732924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12ac892aa6b4154b90e992217298624)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:20.735821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:20.805627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12ac892aa6b4154b90e992217298624', 'request_id': 'req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:20.824242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12ac892aa6b4154b90e992217298624', 'request_id': 'req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:20.833093 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12ac892aa6b4154b90e992217298624', 'request_id': 'req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:20.845352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] The token cache is being invalidate because project c12ac892aa6b4154b90e992217298624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:20.845657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:20.846292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:20.851864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1337/2640] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:20 2022] DELETE /identity/v3/projects/c12ac892aa6b4154b90e992217298624 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:21.041846 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25b643-7b23-4184-87dc-443bffd662e0 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:21.044864 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-db25b643-7b23-4184-87dc-443bffd662e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:21.067173 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25b643-7b23-4184-87dc-443bffd662e0 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:21.133879 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186093b-1883-4e5a-aa72-2d0a00faed5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:21.134571 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186093b-1883-4e5a-aa72-2d0a00faed5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:21.134879 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186093b-1883-4e5a-aa72-2d0a00faed5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:21.135153 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186093b-1883-4e5a-aa72-2d0a00faed5d admin admin] PATH_INFO: `/v3/users/09b055bb15ce4b2ab4276b0090e16122` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:21.136317 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186093b-1883-4e5a-aa72-2d0a00faed5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b055bb15ce4b2ab4276b0090e16122)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:21.138269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186093b-1883-4e5a-aa72-2d0a00faed5d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:21.185863 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a186093b-1883-4e5a-aa72-2d0a00faed5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b055bb15ce4b2ab4276b0090e16122'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:21.192735 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a186093b-1883-4e5a-aa72-2d0a00faed5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b055bb15ce4b2ab4276b0090e16122'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:21.193691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a186093b-1883-4e5a-aa72-2d0a00faed5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b055bb15ce4b2ab4276b0090e16122'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:21.194307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a186093b-1883-4e5a-aa72-2d0a00faed5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b055bb15ce4b2ab4276b0090e16122'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:21.205367 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:21.208829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a186093b-1883-4e5a-aa72-2d0a00faed5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b055bb15ce4b2ab4276b0090e16122'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:21.224095 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1304/2641] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:21 2022] DELETE /identity/v3/users/09b055bb15ce4b2ab4276b0090e16122 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:21.412133 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:21.413949 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:21.431745 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b5506-b29a-4c3f-abc1-9f83a74f3e0c admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:21.490584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:21.491390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:21.491669 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:21.491973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88 admin admin] PATH_INFO: `/v3/projects/594f04dadf2a45fcb5fe2a80744cb9e6` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:21.493134 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=594f04dadf2a45fcb5fe2a80744cb9e6)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:21.495083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:21.517871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:21.544024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '594f04dadf2a45fcb5fe2a80744cb9e6', 'request_id': 'req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:21.549509 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '594f04dadf2a45fcb5fe2a80744cb9e6', 'request_id': 'req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:21.557427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '594f04dadf2a45fcb5fe2a80744cb9e6', 'request_id': 'req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:21.568608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88 admin admin] The token cache is being invalidate because project 594f04dadf2a45fcb5fe2a80744cb9e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:21.568892 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:21.569524 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-f0bdeebb-fdbf-4724-a4a5-4abe2ae13b88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:21.571009 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1338/2642] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:21 2022] DELETE /identity/v3/projects/594f04dadf2a45fcb5fe2a80744cb9e6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:21.575905 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8b203-72ba-4d00-a728-af794a6b2f2c None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:21.576198 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8b203-72ba-4d00-a728-af794a6b2f2c None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:21.576478 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8b203-72ba-4d00-a728-af794a6b2f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:21.595647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a4b8b203-72ba-4d00-a728-af794a6b2f2c None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:21.596974 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a4b8b203-72ba-4d00-a728-af794a6b2f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:21.658639 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1305/2643] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:21.667583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30352e-9dc1-4e24-a26f-66b4c43dcaff None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:21.667899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30352e-9dc1-4e24-a26f-66b4c43dcaff None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:21.668221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30352e-9dc1-4e24-a26f-66b4c43dcaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:21.704345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-6c30352e-9dc1-4e24-a26f-66b4c43dcaff None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:21.706240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6c30352e-9dc1-4e24-a26f-66b4c43dcaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:21.717046 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1339/2644] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:21.718842 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8b203-72ba-4d00-a728-af794a6b2f2c None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:21.730132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8b203-72ba-4d00-a728-af794a6b2f2c None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:21.734819 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dbbf1-b51a-49f8-a80f-ca12f61d0572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:21.735431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dbbf1-b51a-49f8-a80f-ca12f61d0572 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:21.735739 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dbbf1-b51a-49f8-a80f-ca12f61d0572 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:21.736429 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dbbf1-b51a-49f8-a80f-ca12f61d0572 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:21.736670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dbbf1-b51a-49f8-a80f-ca12f61d0572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:21.737018 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dbbf1-b51a-49f8-a80f-ca12f61d0572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:21.739605 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dbbf1-b51a-49f8-a80f-ca12f61d0572 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:21.752373 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1306/2645] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:59:21.756720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30352e-9dc1-4e24-a26f-66b4c43dcaff None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:21.766330 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30352e-9dc1-4e24-a26f-66b4c43dcaff None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:21.771569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f6dfd-01cb-4a22-93c5-e52f4e410828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:21.772202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f6dfd-01cb-4a22-93c5-e52f4e410828 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:21.772478 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f6dfd-01cb-4a22-93c5-e52f4e410828 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:21.773259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f6dfd-01cb-4a22-93c5-e52f4e410828 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:21.773384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f6dfd-01cb-4a22-93c5-e52f4e410828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:21.775256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f6dfd-01cb-4a22-93c5-e52f4e410828 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:21.805176 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1340/2646] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:21 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:59:21.809869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dbbf1-b51a-49f8-a80f-ca12f61d0572 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:21.821311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dbbf1-b51a-49f8-a80f-ca12f61d0572 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:21.881795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16b285-2f0b-4942-8e94-619ee5c4a7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:21.882412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16b285-2f0b-4942-8e94-619ee5c4a7e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:21.882693 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16b285-2f0b-4942-8e94-619ee5c4a7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:21.882971 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16b285-2f0b-4942-8e94-619ee5c4a7e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:21.883596 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16b285-2f0b-4942-8e94-619ee5c4a7e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:21.885690 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16b285-2f0b-4942-8e94-619ee5c4a7e5 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:21.915849 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1307/2647] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:21 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:21.922433 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f6dfd-01cb-4a22-93c5-e52f4e410828 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:21.931571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f6dfd-01cb-4a22-93c5-e52f4e410828 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:21.937251 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eca4d8-407a-416b-99b5-0ef3479b2f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:21.937904 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eca4d8-407a-416b-99b5-0ef3479b2f7f admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:21.938381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eca4d8-407a-416b-99b5-0ef3479b2f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:21.938652 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eca4d8-407a-416b-99b5-0ef3479b2f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:21.939294 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eca4d8-407a-416b-99b5-0ef3479b2f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:21.940985 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eca4d8-407a-416b-99b5-0ef3479b2f7f admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:21.955781 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1341/2648] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:21.957700 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16b285-2f0b-4942-8e94-619ee5c4a7e5 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:21.967977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16b285-2f0b-4942-8e94-619ee5c4a7e5 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:21.973359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe6fbf-9883-46b1-a92a-feb16f5df95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:21.973978 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe6fbf-9883-46b1-a92a-feb16f5df95d admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:21.974262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe6fbf-9883-46b1-a92a-feb16f5df95d admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:21.974539 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe6fbf-9883-46b1-a92a-feb16f5df95d admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:21.975178 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe6fbf-9883-46b1-a92a-feb16f5df95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:21.976807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe6fbf-9883-46b1-a92a-feb16f5df95d admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:21.987484 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1308/2649] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:21.992429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eca4d8-407a-416b-99b5-0ef3479b2f7f admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:22.002560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eca4d8-407a-416b-99b5-0ef3479b2f7f admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:22.007419 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90486dda-6130-4bcf-874b-8b0d8c5520cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:22.008106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90486dda-6130-4bcf-874b-8b0d8c5520cd admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:22.008431 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90486dda-6130-4bcf-874b-8b0d8c5520cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:22.008700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90486dda-6130-4bcf-874b-8b0d8c5520cd admin admin] PATH_INFO: `/v3/projects/a44459f432d7401cb0da8d29ae94d8d5/users/f23a3b84e0df4cc7ab1d92ca8ecadd13/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:22.025166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90486dda-6130-4bcf-874b-8b0d8c5520cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44459f432d7401cb0da8d29ae94d8d5, user_id=f23a3b84e0df4cc7ab1d92ca8ecadd13, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:22.028082 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90486dda-6130-4bcf-874b-8b0d8c5520cd admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:22.042686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1342/2650] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:21 2022] PUT /identity/v3/projects/a44459f432d7401cb0da8d29ae94d8d5/users/f23a3b84e0df4cc7ab1d92ca8ecadd13/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:22.053533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba07f3f-d825-4357-9cc6-5c2ade9b9cb4 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:22.054052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba07f3f-d825-4357-9cc6-5c2ade9b9cb4 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:22.054161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba07f3f-d825-4357-9cc6-5c2ade9b9cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:22.084758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-1ba07f3f-d825-4357-9cc6-5c2ade9b9cb4 None None] MFA Rules not processed for user `f23a3b84e0df4cc7ab1d92ca8ecadd13`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:22.086050 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1ba07f3f-d825-4357-9cc6-5c2ade9b9cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:22.130697 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:22.166290 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1309/2651] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:22.170846 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90486dda-6130-4bcf-874b-8b0d8c5520cd admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:22.279422 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90486dda-6130-4bcf-874b-8b0d8c5520cd admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:22.284200 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e83fdb-b95e-4fc8-b504-806cd558e28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:22.284799 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e83fdb-b95e-4fc8-b504-806cd558e28c admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:22.285076 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e83fdb-b95e-4fc8-b504-806cd558e28c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:22.285349 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e83fdb-b95e-4fc8-b504-806cd558e28c admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:22.285973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e83fdb-b95e-4fc8-b504-806cd558e28c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:22.287914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e83fdb-b95e-4fc8-b504-806cd558e28c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:22.297208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:22.330202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1343/2652] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:22 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:22.335102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba07f3f-d825-4357-9cc6-5c2ade9b9cb4 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:22.361680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba07f3f-d825-4357-9cc6-5c2ade9b9cb4 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:22.419154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022af19a-0e7e-479b-ab8b-2febf1627b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:22.419773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022af19a-0e7e-479b-ab8b-2febf1627b54 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:22.420052 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022af19a-0e7e-479b-ab8b-2febf1627b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:22.420324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022af19a-0e7e-479b-ab8b-2febf1627b54 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:22.420975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022af19a-0e7e-479b-ab8b-2febf1627b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:22.422868 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022af19a-0e7e-479b-ab8b-2febf1627b54 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:22.444016 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1310/2653] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:22 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:22.448064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e83fdb-b95e-4fc8-b504-806cd558e28c admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:22.458231 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e83fdb-b95e-4fc8-b504-806cd558e28c admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:22.462782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14287287-9404-4f9b-9e6e-daa197f76655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:22.463373 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14287287-9404-4f9b-9e6e-daa197f76655 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:22.463647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14287287-9404-4f9b-9e6e-daa197f76655 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:22.464123 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14287287-9404-4f9b-9e6e-daa197f76655 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:22.464761 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14287287-9404-4f9b-9e6e-daa197f76655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:22.466339 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14287287-9404-4f9b-9e6e-daa197f76655 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:22.476877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1344/2654] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:22.481327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022af19a-0e7e-479b-ab8b-2febf1627b54 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:22.494798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022af19a-0e7e-479b-ab8b-2febf1627b54 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:22.500155 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10e952-957c-44af-b328-5f3f3a1fe39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:22.500817 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10e952-957c-44af-b328-5f3f3a1fe39c admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:22.501097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10e952-957c-44af-b328-5f3f3a1fe39c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:22.501370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10e952-957c-44af-b328-5f3f3a1fe39c admin admin] PATH_INFO: `/v3/projects/08f90b3937c54cd6b23e82393715f4cb/users/e8a8ae659c354eb8a311e88bab44b9ae/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:22.517946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10e952-957c-44af-b328-5f3f3a1fe39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f90b3937c54cd6b23e82393715f4cb, user_id=e8a8ae659c354eb8a311e88bab44b9ae, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:22.521469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10e952-957c-44af-b328-5f3f3a1fe39c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:22.543748 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1311/2655] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:22 2022] PUT /identity/v3/projects/08f90b3937c54cd6b23e82393715f4cb/users/e8a8ae659c354eb8a311e88bab44b9ae/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:22.547808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14287287-9404-4f9b-9e6e-daa197f76655 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:22.558551 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14287287-9404-4f9b-9e6e-daa197f76655 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:22.614002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d6edb-2017-4211-b942-30988947fd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:22.614605 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d6edb-2017-4211-b942-30988947fd27 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:22.614884 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d6edb-2017-4211-b942-30988947fd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:22.615180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d6edb-2017-4211-b942-30988947fd27 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:22.615872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d6edb-2017-4211-b942-30988947fd27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:22.617525 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d6edb-2017-4211-b942-30988947fd27 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:22.627460 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1345/2656] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:22.631782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10e952-957c-44af-b328-5f3f3a1fe39c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:22.642251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10e952-957c-44af-b328-5f3f3a1fe39c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:22.647359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21aa03f-8aa5-4b44-a06e-9a881f4ccdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:22.648035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21aa03f-8aa5-4b44-a06e-9a881f4ccdec admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:22.648318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21aa03f-8aa5-4b44-a06e-9a881f4ccdec admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:22.648594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21aa03f-8aa5-4b44-a06e-9a881f4ccdec admin admin] PATH_INFO: `/v3/projects/08f90b3937c54cd6b23e82393715f4cb/users/e8a8ae659c354eb8a311e88bab44b9ae/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:22.650556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21aa03f-8aa5-4b44-a06e-9a881f4ccdec admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f90b3937c54cd6b23e82393715f4cb, user_id=e8a8ae659c354eb8a311e88bab44b9ae, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:22.653226 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21aa03f-8aa5-4b44-a06e-9a881f4ccdec admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:22.663818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1312/2657] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:22 2022] PUT /identity/v3/projects/08f90b3937c54cd6b23e82393715f4cb/users/e8a8ae659c354eb8a311e88bab44b9ae/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:22.675219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc780edb-ae63-454a-971a-fff72a88a5af None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:22.675497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc780edb-ae63-454a-971a-fff72a88a5af None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:22.675806 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc780edb-ae63-454a-971a-fff72a88a5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:22.704262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-fc780edb-ae63-454a-971a-fff72a88a5af None None] MFA Rules not processed for user `e8a8ae659c354eb8a311e88bab44b9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:22.705474 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-fc780edb-ae63-454a-971a-fff72a88a5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:22.780505 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1346/2658] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:22.792690 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21aa03f-8aa5-4b44-a06e-9a881f4ccdec admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:22.794646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a21aa03f-8aa5-4b44-a06e-9a881f4ccdec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:22.871533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21aa03f-8aa5-4b44-a06e-9a881f4ccdec admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:22.875278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041b5c8-30d7-4df8-850a-221558b4983e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:22.875947 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b5c8-30d7-4df8-850a-221558b4983e service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:22.876261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b5c8-30d7-4df8-850a-221558b4983e service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:22.876555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b5c8-30d7-4df8-850a-221558b4983e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:22.884396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:22.886648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041b5c8-30d7-4df8-850a-221558b4983e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:22.888548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041b5c8-30d7-4df8-850a-221558b4983e service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:22.900163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1313/2659] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:22.966116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc780edb-ae63-454a-971a-fff72a88a5af None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:23.018572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:23.033794 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc780edb-ae63-454a-971a-fff72a88a5af None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:23.038921 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:23.039547 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:23.039870 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:23.040141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] PATH_INFO: `/v3/users/f23a3b84e0df4cc7ab1d92ca8ecadd13` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:23.041279 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f23a3b84e0df4cc7ab1d92ca8ecadd13)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:23.043209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:23.090754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f23a3b84e0df4cc7ab1d92ca8ecadd13'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.098784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f23a3b84e0df4cc7ab1d92ca8ecadd13'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.099623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f23a3b84e0df4cc7ab1d92ca8ecadd13'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.114712 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f23a3b84e0df4cc7ab1d92ca8ecadd13'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.121025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f23a3b84e0df4cc7ab1d92ca8ecadd13'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.127220 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1347/2660] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:22 2022] DELETE /identity/v3/users/f23a3b84e0df4cc7ab1d92ca8ecadd13 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:23.132026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d83b9-3dd4-44bc-81e2-ee6893798a3b None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:23.132523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d83b9-3dd4-44bc-81e2-ee6893798a3b None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:23.132636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d83b9-3dd4-44bc-81e2-ee6893798a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:23.161078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-821d83b9-3dd4-44bc-81e2-ee6893798a3b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:23.162276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-821d83b9-3dd4-44bc-81e2-ee6893798a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:23.223720 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1314/2661] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:23.233770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:23.235205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:23.297110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c34e9-d7c0-406a-be4e-23f95cea9c33 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:23.301856 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b386c26-b95d-4952-be1f-ee4bcbf904f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:23.302463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b386c26-b95d-4952-be1f-ee4bcbf904f7 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:23.302736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b386c26-b95d-4952-be1f-ee4bcbf904f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:23.303004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b386c26-b95d-4952-be1f-ee4bcbf904f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:23.315310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b386c26-b95d-4952-be1f-ee4bcbf904f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:23.317679 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b386c26-b95d-4952-be1f-ee4bcbf904f7 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:23.328018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1348/2662] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:23.621262 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d83b9-3dd4-44bc-81e2-ee6893798a3b None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:23.624210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-821d83b9-3dd4-44bc-81e2-ee6893798a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:23.648568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d83b9-3dd4-44bc-81e2-ee6893798a3b None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:23.666236 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:23.666887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:23.667169 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:23.667446 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] PATH_INFO: `/v3/projects/a44459f432d7401cb0da8d29ae94d8d5` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:23.668632 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a44459f432d7401cb0da8d29ae94d8d5)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:23.670554 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:23.735609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a44459f432d7401cb0da8d29ae94d8d5', 'request_id': 'req-7847999b-8f5b-4d0b-9553-045566d9ad46'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.741940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a44459f432d7401cb0da8d29ae94d8d5', 'request_id': 'req-7847999b-8f5b-4d0b-9553-045566d9ad46'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.749333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a44459f432d7401cb0da8d29ae94d8d5', 'request_id': 'req-7847999b-8f5b-4d0b-9553-045566d9ad46'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.760866 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] The token cache is being invalidate because project a44459f432d7401cb0da8d29ae94d8d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:23.761187 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.761885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.763508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1315/2663] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:23 2022] DELETE /identity/v3/projects/a44459f432d7401cb0da8d29ae94d8d5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:23.767986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b386c26-b95d-4952-be1f-ee4bcbf904f7 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:23.778544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b386c26-b95d-4952-be1f-ee4bcbf904f7 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:23.842381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4562280-4540-49e2-bef3-1b7e57791afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:23.843205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4562280-4540-49e2-bef3-1b7e57791afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:23.843865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4562280-4540-49e2-bef3-1b7e57791afd admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:23.844345 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4562280-4540-49e2-bef3-1b7e57791afd admin admin] PATH_INFO: `/v3/users/e8a8ae659c354eb8a311e88bab44b9ae` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:23.846221 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4562280-4540-49e2-bef3-1b7e57791afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8a8ae659c354eb8a311e88bab44b9ae)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:23.848784 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4562280-4540-49e2-bef3-1b7e57791afd admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:23.862808 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:23.914293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4562280-4540-49e2-bef3-1b7e57791afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8a8ae659c354eb8a311e88bab44b9ae'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.924078 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4562280-4540-49e2-bef3-1b7e57791afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8a8ae659c354eb8a311e88bab44b9ae'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.925037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4562280-4540-49e2-bef3-1b7e57791afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8a8ae659c354eb8a311e88bab44b9ae'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.940440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4562280-4540-49e2-bef3-1b7e57791afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8a8ae659c354eb8a311e88bab44b9ae'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.948674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4562280-4540-49e2-bef3-1b7e57791afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8a8ae659c354eb8a311e88bab44b9ae'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:23.955250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1349/2664] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:23 2022] DELETE /identity/v3/users/e8a8ae659c354eb8a311e88bab44b9ae => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:24.246161 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:24.247904 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:24.263055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847999b-8f5b-4d0b-9553-045566d9ad46 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:24.322770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647de16-7d82-4759-b2cc-ea26753441bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:24.323406 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647de16-7d82-4759-b2cc-ea26753441bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:24.323700 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647de16-7d82-4759-b2cc-ea26753441bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:24.323981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647de16-7d82-4759-b2cc-ea26753441bd admin admin] PATH_INFO: `/v3/projects/08f90b3937c54cd6b23e82393715f4cb` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:24.325105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647de16-7d82-4759-b2cc-ea26753441bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f90b3937c54cd6b23e82393715f4cb)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:24.327111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647de16-7d82-4759-b2cc-ea26753441bd admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:24.343203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:24.373662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0647de16-7d82-4759-b2cc-ea26753441bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f90b3937c54cd6b23e82393715f4cb', 'request_id': 'req-0647de16-7d82-4759-b2cc-ea26753441bd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:24.379745 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0647de16-7d82-4759-b2cc-ea26753441bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f90b3937c54cd6b23e82393715f4cb', 'request_id': 'req-0647de16-7d82-4759-b2cc-ea26753441bd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:24.387103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0647de16-7d82-4759-b2cc-ea26753441bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f90b3937c54cd6b23e82393715f4cb', 'request_id': 'req-0647de16-7d82-4759-b2cc-ea26753441bd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:24.398436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0647de16-7d82-4759-b2cc-ea26753441bd admin admin] The token cache is being invalidate because project 08f90b3937c54cd6b23e82393715f4cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:24.398722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0647de16-7d82-4759-b2cc-ea26753441bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:24.399391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-0647de16-7d82-4759-b2cc-ea26753441bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:24.400851 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1316/2665] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:24 2022] DELETE /identity/v3/projects/08f90b3937c54cd6b23e82393715f4cb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:24.405971 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbea369-ace3-41e9-beb3-9f91da51abe2 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:24.406283 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbea369-ace3-41e9-beb3-9f91da51abe2 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:24.406560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbea369-ace3-41e9-beb3-9f91da51abe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:24.456574 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-dbbea369-ace3-41e9-beb3-9f91da51abe2 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:24.457810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-dbbea369-ace3-41e9-beb3-9f91da51abe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:24.517709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1350/2666] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:24.525421 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6937652-0f40-4abd-a370-f9053bf8a47e None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:24.525727 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6937652-0f40-4abd-a370-f9053bf8a47e None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:24.526007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6937652-0f40-4abd-a370-f9053bf8a47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:24.545949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-d6937652-0f40-4abd-a370-f9053bf8a47e None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:24.547159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d6937652-0f40-4abd-a370-f9053bf8a47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:24.555921 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1317/2667] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:24.560091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbea369-ace3-41e9-beb3-9f91da51abe2 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:24.569750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbea369-ace3-41e9-beb3-9f91da51abe2 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:24.574229 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bfdd9-464a-4f5f-817d-9271f58abb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:24.574840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bfdd9-464a-4f5f-817d-9271f58abb02 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:24.575116 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bfdd9-464a-4f5f-817d-9271f58abb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:24.575385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bfdd9-464a-4f5f-817d-9271f58abb02 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:24.576048 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bfdd9-464a-4f5f-817d-9271f58abb02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:24.576372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bfdd9-464a-4f5f-817d-9271f58abb02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:24.577952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bfdd9-464a-4f5f-817d-9271f58abb02 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:24.589463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1351/2668] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:59:24.593848 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6937652-0f40-4abd-a370-f9053bf8a47e None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:24.603474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6937652-0f40-4abd-a370-f9053bf8a47e None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:24.608245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b7225-63ca-4f72-a34a-488fb18da317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:24.608833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b7225-63ca-4f72-a34a-488fb18da317 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:24.609114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b7225-63ca-4f72-a34a-488fb18da317 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:24.609389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b7225-63ca-4f72-a34a-488fb18da317 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:24.610009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b7225-63ca-4f72-a34a-488fb18da317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:24.612006 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b7225-63ca-4f72-a34a-488fb18da317 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:24.639300 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1318/2669] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:24 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:24.643466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bfdd9-464a-4f5f-817d-9271f58abb02 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:24.654603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bfdd9-464a-4f5f-817d-9271f58abb02 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:24.703381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:24.714826 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f98d12-32a7-4bd7-960d-e3d633848ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:24.715372 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f98d12-32a7-4bd7-960d-e3d633848ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:24.715650 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f98d12-32a7-4bd7-960d-e3d633848ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:24.715946 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f98d12-32a7-4bd7-960d-e3d633848ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:24.716854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f98d12-32a7-4bd7-960d-e3d633848ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:24.718756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f98d12-32a7-4bd7-960d-e3d633848ec7 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:24.740586 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b7225-63ca-4f72-a34a-488fb18da317 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:24.742296 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bd6b7225-63ca-4f72-a34a-488fb18da317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:24.743481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1352/2670] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:24 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:59:24.747197 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f98d12-32a7-4bd7-960d-e3d633848ec7 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:24.755034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b7225-63ca-4f72-a34a-488fb18da317 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:24.756960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f98d12-32a7-4bd7-960d-e3d633848ec7 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:24.759747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d289290-bc23-48c3-bf8f-e2f4d56e26ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:24.760392 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d289290-bc23-48c3-bf8f-e2f4d56e26ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:24.760653 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d289290-bc23-48c3-bf8f-e2f4d56e26ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:24.760925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d289290-bc23-48c3-bf8f-e2f4d56e26ac admin admin] PATH_INFO: `/v3/users/b467ab02084b482a8ce39471e4381dfd` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:24.761752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d09e92-4d5b-41e4-b0e3-da7c31a601f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:24.762057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d289290-bc23-48c3-bf8f-e2f4d56e26ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=b467ab02084b482a8ce39471e4381dfd)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:24.762330 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d09e92-4d5b-41e4-b0e3-da7c31a601f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:24.762608 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d09e92-4d5b-41e4-b0e3-da7c31a601f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:24.762894 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d09e92-4d5b-41e4-b0e3-da7c31a601f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:24.763521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d09e92-4d5b-41e4-b0e3-da7c31a601f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:24.764261 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d289290-bc23-48c3-bf8f-e2f4d56e26ac admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:24.765557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d09e92-4d5b-41e4-b0e3-da7c31a601f5 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:24.777155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1353/2671] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:24.781267 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d09e92-4d5b-41e4-b0e3-da7c31a601f5 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:24.792247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d09e92-4d5b-41e4-b0e3-da7c31a601f5 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:24.797099 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65596766-3899-42a9-a572-cc38fc2cecbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:24.797687 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65596766-3899-42a9-a572-cc38fc2cecbb admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:24.798142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65596766-3899-42a9-a572-cc38fc2cecbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:24.798393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65596766-3899-42a9-a572-cc38fc2cecbb admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:24.799024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65596766-3899-42a9-a572-cc38fc2cecbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:24.800646 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65596766-3899-42a9-a572-cc38fc2cecbb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:24.811110 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1354/2672] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:24.814690 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2d289290-bc23-48c3-bf8f-e2f4d56e26ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b467ab02084b482a8ce39471e4381dfd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:24.816139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65596766-3899-42a9-a572-cc38fc2cecbb admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:24.823474 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2d289290-bc23-48c3-bf8f-e2f4d56e26ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b467ab02084b482a8ce39471e4381dfd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:24.824349 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2d289290-bc23-48c3-bf8f-e2f4d56e26ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b467ab02084b482a8ce39471e4381dfd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:24.825199 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2d289290-bc23-48c3-bf8f-e2f4d56e26ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b467ab02084b482a8ce39471e4381dfd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:24.826364 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65596766-3899-42a9-a572-cc38fc2cecbb admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:24.831820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fccc2-a358-42b9-a82d-fe9fae6d71cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:24.833019 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fccc2-a358-42b9-a82d-fe9fae6d71cb admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:24.833294 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fccc2-a358-42b9-a82d-fe9fae6d71cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:24.833564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fccc2-a358-42b9-a82d-fe9fae6d71cb admin admin] PATH_INFO: `/v3/projects/19c522cfb1cf4d20a8579c13fa581059/users/1f5f939a10574c1a9614e3cdd6432a62/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:24.839427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2d289290-bc23-48c3-bf8f-e2f4d56e26ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b467ab02084b482a8ce39471e4381dfd'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:24.850242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fccc2-a358-42b9-a82d-fe9fae6d71cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c522cfb1cf4d20a8579c13fa581059, user_id=1f5f939a10574c1a9614e3cdd6432a62, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:24.853136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fccc2-a358-42b9-a82d-fe9fae6d71cb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:24.854111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1319/2673] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:24 2022] DELETE /identity/v3/users/b467ab02084b482a8ce39471e4381dfd => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:24.858816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405b073-708d-46d5-b0d9-48415a3a7edb None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:24.859157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405b073-708d-46d5-b0d9-48415a3a7edb None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:24.859503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405b073-708d-46d5-b0d9-48415a3a7edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:24.866847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1355/2674] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:24 2022] PUT /identity/v3/projects/19c522cfb1cf4d20a8579c13fa581059/users/1f5f939a10574c1a9614e3cdd6432a62/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:24.877244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e9ce-ebd4-4439-bd65-a9088054483d None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:24.877524 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e9ce-ebd4-4439-bd65-a9088054483d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:24.877790 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e9ce-ebd4-4439-bd65-a9088054483d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:24.880231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-1405b073-708d-46d5-b0d9-48415a3a7edb None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:24.881433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1405b073-708d-46d5-b0d9-48415a3a7edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:24.908166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-cc79e9ce-ebd4-4439-bd65-a9088054483d None None] MFA Rules not processed for user `1f5f939a10574c1a9614e3cdd6432a62`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:24.909880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-cc79e9ce-ebd4-4439-bd65-a9088054483d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:24.925431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:24.946633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1320/2675] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:24.956332 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405b073-708d-46d5-b0d9-48415a3a7edb None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:24.958221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1405b073-708d-46d5-b0d9-48415a3a7edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:24.987290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1356/2676] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:24.991951 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79e9ce-ebd4-4439-bd65-a9088054483d None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:24.993273 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-cc79e9ce-ebd4-4439-bd65-a9088054483d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:25.007747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79e9ce-ebd4-4439-bd65-a9088054483d None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.012674 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec953ea-2c28-472a-8313-5f3ea4dd224a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.013566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec953ea-2c28-472a-8313-5f3ea4dd224a admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.014847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec953ea-2c28-472a-8313-5f3ea4dd224a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.014847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec953ea-2c28-472a-8313-5f3ea4dd224a admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.015064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec953ea-2c28-472a-8313-5f3ea4dd224a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.017005 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec953ea-2c28-472a-8313-5f3ea4dd224a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.018434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405b073-708d-46d5-b0d9-48415a3a7edb None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.022943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8c8f2-8101-4f0c-836f-80e85c4b1cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.023562 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8c8f2-8101-4f0c-836f-80e85c4b1cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.027133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8c8f2-8101-4f0c-836f-80e85c4b1cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.027133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8c8f2-8101-4f0c-836f-80e85c4b1cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.034409 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8c8f2-8101-4f0c-836f-80e85c4b1cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.036545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8c8f2-8101-4f0c-836f-80e85c4b1cc8 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.048601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1321/2677] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:59:25.058713 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1357/2678] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:24 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:59:25.059827 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8c8f2-8101-4f0c-836f-80e85c4b1cc8 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.070962 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8c8f2-8101-4f0c-836f-80e85c4b1cc8 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.141327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d8bd8-a922-4597-a823-d66581efec20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.141941 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d8bd8-a922-4597-a823-d66581efec20 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.142215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d8bd8-a922-4597-a823-d66581efec20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.142486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d8bd8-a922-4597-a823-d66581efec20 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.143106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d8bd8-a922-4597-a823-d66581efec20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.145625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d8bd8-a922-4597-a823-d66581efec20 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.170278 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1322/2679] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:25 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:59:25.174633 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec953ea-2c28-472a-8313-5f3ea4dd224a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.185411 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec953ea-2c28-472a-8313-5f3ea4dd224a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.190966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2850c0c-8679-4975-a829-ef1a271baa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.191281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2850c0c-8679-4975-a829-ef1a271baa48 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.192328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2850c0c-8679-4975-a829-ef1a271baa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.192671 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2850c0c-8679-4975-a829-ef1a271baa48 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.193384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2850c0c-8679-4975-a829-ef1a271baa48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.195022 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2850c0c-8679-4975-a829-ef1a271baa48 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.203882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d8bd8-a922-4597-a823-d66581efec20 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.212856 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-e53d8bd8-a922-4597-a823-d66581efec20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:25.217401 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1358/2680] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:25.222040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2850c0c-8679-4975-a829-ef1a271baa48 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.230599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d8bd8-a922-4597-a823-d66581efec20 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.233025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2850c0c-8679-4975-a829-ef1a271baa48 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.236110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6058456-516c-47e2-b5d7-10dcb4830462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.236757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6058456-516c-47e2-b5d7-10dcb4830462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.237039 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6058456-516c-47e2-b5d7-10dcb4830462 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.237368 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6058456-516c-47e2-b5d7-10dcb4830462 admin admin] PATH_INFO: `/v3/users/9e36ea7f5f1542118bb9dc1efe78c42c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.238520 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6058456-516c-47e2-b5d7-10dcb4830462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e36ea7f5f1542118bb9dc1efe78c42c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.239219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96366ac5-7095-4c33-98e5-d7daebd88023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.239909 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96366ac5-7095-4c33-98e5-d7daebd88023 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.240189 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96366ac5-7095-4c33-98e5-d7daebd88023 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.240466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96366ac5-7095-4c33-98e5-d7daebd88023 admin admin] PATH_INFO: `/v3/projects/ab0c871d41294ac4ba92df0913346fef/users/4094a747725d4078b651a7cbe0e666ae/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.240582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6058456-516c-47e2-b5d7-10dcb4830462 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.257510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96366ac5-7095-4c33-98e5-d7daebd88023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0c871d41294ac4ba92df0913346fef, user_id=4094a747725d4078b651a7cbe0e666ae, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.260201 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96366ac5-7095-4c33-98e5-d7daebd88023 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.280381 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1359/2681] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:25 2022] PUT /identity/v3/projects/ab0c871d41294ac4ba92df0913346fef/users/4094a747725d4078b651a7cbe0e666ae/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:25.282062 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96366ac5-7095-4c33-98e5-d7daebd88023 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.294972 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a6058456-516c-47e2-b5d7-10dcb4830462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e36ea7f5f1542118bb9dc1efe78c42c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:25.303229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a6058456-516c-47e2-b5d7-10dcb4830462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e36ea7f5f1542118bb9dc1efe78c42c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:25.304114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a6058456-516c-47e2-b5d7-10dcb4830462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e36ea7f5f1542118bb9dc1efe78c42c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:25.304830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a6058456-516c-47e2-b5d7-10dcb4830462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e36ea7f5f1542118bb9dc1efe78c42c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:25.321324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:25.322294 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-a6058456-516c-47e2-b5d7-10dcb4830462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e36ea7f5f1542118bb9dc1efe78c42c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:25.325469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:25.344090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1323/2682] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:25 2022] DELETE /identity/v3/users/9e36ea7f5f1542118bb9dc1efe78c42c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:25.350674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a833711-578c-442a-96ca-fe823cb9e345 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.350674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a833711-578c-442a-96ca-fe823cb9e345 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.350674 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a833711-578c-442a-96ca-fe823cb9e345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.368567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96366ac5-7095-4c33-98e5-d7daebd88023 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.372784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-1a833711-578c-442a-96ca-fe823cb9e345 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:25.372784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1a833711-578c-442a-96ca-fe823cb9e345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:25.436353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1324/2683] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:25.437194 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a833711-578c-442a-96ca-fe823cb9e345 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.438434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1a833711-578c-442a-96ca-fe823cb9e345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:25.443055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d96bbe-ec18-428a-b1d3-458993be1ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.443671 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d96bbe-ec18-428a-b1d3-458993be1ddc admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.443980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d96bbe-ec18-428a-b1d3-458993be1ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.444256 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d96bbe-ec18-428a-b1d3-458993be1ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.445030 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d96bbe-ec18-428a-b1d3-458993be1ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.446868 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d96bbe-ec18-428a-b1d3-458993be1ddc admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.450691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a833711-578c-442a-96ca-fe823cb9e345 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.455307 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.455949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.456222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.456490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] PATH_INFO: `/v3/projects/76ecb6d53b5146bb98eebef2718373ec` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.457179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1360/2684] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:25.458172 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ecb6d53b5146bb98eebef2718373ec)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.460136 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.460647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d96bbe-ec18-428a-b1d3-458993be1ddc admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.462684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-26d96bbe-ec18-428a-b1d3-458993be1ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:25.509525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ecb6d53b5146bb98eebef2718373ec', 'request_id': 'req-ee75114f-54d6-4235-94a8-43baac891f12'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:25.515530 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ecb6d53b5146bb98eebef2718373ec', 'request_id': 'req-ee75114f-54d6-4235-94a8-43baac891f12'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:25.523330 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d96bbe-ec18-428a-b1d3-458993be1ddc admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.523521 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ecb6d53b5146bb98eebef2718373ec', 'request_id': 'req-ee75114f-54d6-4235-94a8-43baac891f12'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:25.534627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] The token cache is being invalidate because project 76ecb6d53b5146bb98eebef2718373ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:25.534911 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:25.535612 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:25.537557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1325/2685] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:25 2022] DELETE /identity/v3/projects/76ecb6d53b5146bb98eebef2718373ec => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:25.537907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.539202 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:25.576341 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca3c7a-dc3a-45a8-8c6f-a81621de3c99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.577054 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca3c7a-dc3a-45a8-8c6f-a81621de3c99 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.577312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca3c7a-dc3a-45a8-8c6f-a81621de3c99 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.577577 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca3c7a-dc3a-45a8-8c6f-a81621de3c99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.579477 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-41ca3c7a-dc3a-45a8-8c6f-a81621de3c99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:25.603505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75114f-54d6-4235-94a8-43baac891f12 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.608244 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ebbe5-ca6f-47b0-825d-e6a2d671bcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.608907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ebbe5-ca6f-47b0-825d-e6a2d671bcbb admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.609183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ebbe5-ca6f-47b0-825d-e6a2d671bcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.609459 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ebbe5-ca6f-47b0-825d-e6a2d671bcbb admin admin] PATH_INFO: `/v3/projects/ab0c871d41294ac4ba92df0913346fef/users/4094a747725d4078b651a7cbe0e666ae/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.611401 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ebbe5-ca6f-47b0-825d-e6a2d671bcbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0c871d41294ac4ba92df0913346fef, user_id=4094a747725d4078b651a7cbe0e666ae, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.614433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ebbe5-ca6f-47b0-825d-e6a2d671bcbb admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.622699 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:25.624623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1326/2686] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:25 2022] PUT /identity/v3/projects/ab0c871d41294ac4ba92df0913346fef/users/4094a747725d4078b651a7cbe0e666ae/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:25.625826 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3feacd8-d7f5-4949-a012-3a7b1b09b4e3 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.626118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3feacd8-d7f5-4949-a012-3a7b1b09b4e3 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.626389 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3feacd8-d7f5-4949-a012-3a7b1b09b4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.646615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-b3feacd8-d7f5-4949-a012-3a7b1b09b4e3 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:25.646820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca3c7a-dc3a-45a8-8c6f-a81621de3c99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.648131 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca3c7a-dc3a-45a8-8c6f-a81621de3c99 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.648253 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-b3feacd8-d7f5-4949-a012-3a7b1b09b4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:25.652342 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:25.658265 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1361/2687] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:25.659515 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb03a9-e178-4b4a-b3c0-ef1db0fc0833 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.659830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb03a9-e178-4b4a-b3c0-ef1db0fc0833 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.660106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb03a9-e178-4b4a-b3c0-ef1db0fc0833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.693451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-88cb03a9-e178-4b4a-b3c0-ef1db0fc0833 None None] MFA Rules not processed for user `4094a747725d4078b651a7cbe0e666ae`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:25.695311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-88cb03a9-e178-4b4a-b3c0-ef1db0fc0833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:25.720509 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1327/2688] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:25.729063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792dec9-55e5-4322-8454-7db2d0c940a3 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.729378 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792dec9-55e5-4322-8454-7db2d0c940a3 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.729661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792dec9-55e5-4322-8454-7db2d0c940a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.753860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-4792dec9-55e5-4322-8454-7db2d0c940a3 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:25.755434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-4792dec9-55e5-4322-8454-7db2d0c940a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:25.764883 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1328/2689] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:25.769138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792dec9-55e5-4322-8454-7db2d0c940a3 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.778967 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792dec9-55e5-4322-8454-7db2d0c940a3 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.782650 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1362/2690] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:25.790194 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca0af7-6957-49ef-99f9-bfb6e9cf0ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.790825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca0af7-6957-49ef-99f9-bfb6e9cf0ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.791471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca0af7-6957-49ef-99f9-bfb6e9cf0ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.791471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca0af7-6957-49ef-99f9-bfb6e9cf0ab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.792072 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca0af7-6957-49ef-99f9-bfb6e9cf0ab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:25.792371 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca0af7-6957-49ef-99f9-bfb6e9cf0ab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.794221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca0af7-6957-49ef-99f9-bfb6e9cf0ab8 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.795467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb03a9-e178-4b4a-b3c0-ef1db0fc0833 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.797961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-88cb03a9-e178-4b4a-b3c0-ef1db0fc0833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:25.811034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1329/2691] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:59:25.812355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca0af7-6957-49ef-99f9-bfb6e9cf0ab8 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.824569 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca0af7-6957-49ef-99f9-bfb6e9cf0ab8 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.830087 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966abb0a-fa22-4a1c-b42a-26a99c90caa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.830749 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966abb0a-fa22-4a1c-b42a-26a99c90caa9 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.831125 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966abb0a-fa22-4a1c-b42a-26a99c90caa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.831298 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966abb0a-fa22-4a1c-b42a-26a99c90caa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.831964 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966abb0a-fa22-4a1c-b42a-26a99c90caa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.834747 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966abb0a-fa22-4a1c-b42a-26a99c90caa9 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.869635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1330/2692] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:25 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:25.870830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966abb0a-fa22-4a1c-b42a-26a99c90caa9 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.874206 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb03a9-e178-4b4a-b3c0-ef1db0fc0833 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.936649 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40111a68-e36d-4f1a-b4c4-043bd8a3f7c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.937271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40111a68-e36d-4f1a-b4c4-043bd8a3f7c0 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.937549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40111a68-e36d-4f1a-b4c4-043bd8a3f7c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.937821 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40111a68-e36d-4f1a-b4c4-043bd8a3f7c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.946833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966abb0a-fa22-4a1c-b42a-26a99c90caa9 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:25.954936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d7d25-77f6-4bc6-9775-110853acdf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:25.955668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d7d25-77f6-4bc6-9775-110853acdf31 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:25.955976 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d7d25-77f6-4bc6-9775-110853acdf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:25.956250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d7d25-77f6-4bc6-9775-110853acdf31 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:25.956877 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d7d25-77f6-4bc6-9775-110853acdf31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:25.959365 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d7d25-77f6-4bc6-9775-110853acdf31 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:25.983912 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:25.989364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1331/2693] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:25 2022] POST /identity/v3/users => generated 300 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:59:25.990692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d7d25-77f6-4bc6-9775-110853acdf31 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:25.996574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-012d7d25-77f6-4bc6-9775-110853acdf31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:26.010943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d7d25-77f6-4bc6-9775-110853acdf31 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:26.015936 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:26.016504 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:26.016784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:26.017055 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] PATH_INFO: `/v3/projects/d8e090229d414449a449b013410298be` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:26.018177 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e090229d414449a449b013410298be)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:26.020680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:26.023364 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40111a68-e36d-4f1a-b4c4-043bd8a3f7c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:26.024991 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40111a68-e36d-4f1a-b4c4-043bd8a3f7c0 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:26.034919 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1363/2694] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:26.035810 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40111a68-e36d-4f1a-b4c4-043bd8a3f7c0 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:26.047160 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40111a68-e36d-4f1a-b4c4-043bd8a3f7c0 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:26.053244 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ff354-8546-49c2-8f61-17076b42906d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:26.053902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ff354-8546-49c2-8f61-17076b42906d admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:26.054175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ff354-8546-49c2-8f61-17076b42906d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:26.054444 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ff354-8546-49c2-8f61-17076b42906d admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:26.055388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ff354-8546-49c2-8f61-17076b42906d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:26.057109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ff354-8546-49c2-8f61-17076b42906d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:26.070566 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1364/2695] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:26.071893 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e090229d414449a449b013410298be', 'request_id': 'req-2d9f0493-290c-461b-b0cb-8f76d2683a38'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.074063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:26.075885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ff354-8546-49c2-8f61-17076b42906d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:26.079804 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e090229d414449a449b013410298be', 'request_id': 'req-2d9f0493-290c-461b-b0cb-8f76d2683a38'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.089426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ff354-8546-49c2-8f61-17076b42906d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:26.093041 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e090229d414449a449b013410298be', 'request_id': 'req-2d9f0493-290c-461b-b0cb-8f76d2683a38'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.109829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] The token cache is being invalidate because project d8e090229d414449a449b013410298be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:26.110125 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.110822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.115189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1332/2696] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:25 2022] DELETE /identity/v3/projects/d8e090229d414449a449b013410298be => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:26.115189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:26.115189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:26.185938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f0493-290c-461b-b0cb-8f76d2683a38 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:26.191020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7e4b8-f76e-4caf-a2d1-cadab8d26202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:26.192570 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7e4b8-f76e-4caf-a2d1-cadab8d26202 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:26.192949 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7e4b8-f76e-4caf-a2d1-cadab8d26202 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:26.193336 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7e4b8-f76e-4caf-a2d1-cadab8d26202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:26.195594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2ea7e4b8-f76e-4caf-a2d1-cadab8d26202 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:26.278783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7e4b8-f76e-4caf-a2d1-cadab8d26202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:26.280296 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7e4b8-f76e-4caf-a2d1-cadab8d26202 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:26.290130 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1333/2697] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:26.290984 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7e4b8-f76e-4caf-a2d1-cadab8d26202 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:26.294135 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-2ea7e4b8-f76e-4caf-a2d1-cadab8d26202 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:26.295309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23accd00-94cd-4965-aabe-20cf80db13c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:26.297516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23accd00-94cd-4965-aabe-20cf80db13c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:26.298208 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23accd00-94cd-4965-aabe-20cf80db13c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:26.298682 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23accd00-94cd-4965-aabe-20cf80db13c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:26.300224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23accd00-94cd-4965-aabe-20cf80db13c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:26.302937 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23accd00-94cd-4965-aabe-20cf80db13c6 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:26.311141 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7e4b8-f76e-4caf-a2d1-cadab8d26202 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:26.320643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1365/2698] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:26.322295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:26.322999 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:26.323281 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:26.323869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] PATH_INFO: `/v3/users/c23aa2c853964a76832161d32ffa56ea` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:26.325060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c23aa2c853964a76832161d32ffa56ea)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:26.328213 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:26.328613 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23accd00-94cd-4965-aabe-20cf80db13c6 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:26.330618 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-23accd00-94cd-4965-aabe-20cf80db13c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:26.349176 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23accd00-94cd-4965-aabe-20cf80db13c6 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:26.356562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e37eea-daf1-4905-9d72-d9b3b1f10b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:26.357431 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e37eea-daf1-4905-9d72-d9b3b1f10b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:26.357864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e37eea-daf1-4905-9d72-d9b3b1f10b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:26.358506 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e37eea-daf1-4905-9d72-d9b3b1f10b47 admin admin] PATH_INFO: `/v3/projects/622090417eb3426fbbf971444b821e75/users/009059267bbf430c9d592bc9b1271e51/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:26.384227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e37eea-daf1-4905-9d72-d9b3b1f10b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=622090417eb3426fbbf971444b821e75, user_id=009059267bbf430c9d592bc9b1271e51, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:26.388073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e37eea-daf1-4905-9d72-d9b3b1f10b47 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:26.393246 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c23aa2c853964a76832161d32ffa56ea'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.402773 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c23aa2c853964a76832161d32ffa56ea'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.403783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c23aa2c853964a76832161d32ffa56ea'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.404553 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c23aa2c853964a76832161d32ffa56ea'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.407802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1366/2699] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:26 2022] PUT /identity/v3/projects/622090417eb3426fbbf971444b821e75/users/009059267bbf430c9d592bc9b1271e51/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:26.420106 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863620a8-a4b3-4291-961d-135048979cb5 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:26.420748 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863620a8-a4b3-4291-961d-135048979cb5 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:26.421205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863620a8-a4b3-4291-961d-135048979cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:26.426958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c23aa2c853964a76832161d32ffa56ea'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.449993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1334/2700] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:26 2022] DELETE /identity/v3/users/c23aa2c853964a76832161d32ffa56ea => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:26.462732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-863620a8-a4b3-4291-961d-135048979cb5 None None] MFA Rules not processed for user `009059267bbf430c9d592bc9b1271e51`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:26.464227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-863620a8-a4b3-4291-961d-135048979cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:26.579731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1367/2701] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:26 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:26.591469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:26.593267 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:26.605589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:26.693988 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4784c-105d-4671-8211-087620eae035 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:26.699124 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41610b-a7bf-47c9-9738-342ff1840e36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:26.700029 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41610b-a7bf-47c9-9738-342ff1840e36 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:26.700311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41610b-a7bf-47c9-9738-342ff1840e36 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:26.700586 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41610b-a7bf-47c9-9738-342ff1840e36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:26.711889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac41610b-a7bf-47c9-9738-342ff1840e36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:26.714583 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac41610b-a7bf-47c9-9738-342ff1840e36 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:26.726529 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1335/2702] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:26 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:26.776429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863620a8-a4b3-4291-961d-135048979cb5 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:26.778118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-863620a8-a4b3-4291-961d-135048979cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:26.799913 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41610b-a7bf-47c9-9738-342ff1840e36 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:26.802044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ac41610b-a7bf-47c9-9738-342ff1840e36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:26.805900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:26.869712 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863620a8-a4b3-4291-961d-135048979cb5 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:26.874661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32697ceb-af94-480e-8fa1-a834476882c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:26.875320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32697ceb-af94-480e-8fa1-a834476882c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:26.875711 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32697ceb-af94-480e-8fa1-a834476882c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:26.875967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32697ceb-af94-480e-8fa1-a834476882c4 admin admin] PATH_INFO: `/v3/users/1f5f939a10574c1a9614e3cdd6432a62` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:26.877186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32697ceb-af94-480e-8fa1-a834476882c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f5f939a10574c1a9614e3cdd6432a62)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:26.880371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32697ceb-af94-480e-8fa1-a834476882c4 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:26.906522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41610b-a7bf-47c9-9738-342ff1840e36 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:26.911828 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a8877-af3a-4781-a063-ee817fc2fb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:26.912692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a8877-af3a-4781-a063-ee817fc2fb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:26.913030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a8877-af3a-4781-a063-ee817fc2fb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:26.913624 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a8877-af3a-4781-a063-ee817fc2fb1b admin admin] PATH_INFO: `/v3/projects/43e3ed6833064b3e8cf38caa0e7d1cf2` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:26.914860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a8877-af3a-4781-a063-ee817fc2fb1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=43e3ed6833064b3e8cf38caa0e7d1cf2)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:26.917168 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a8877-af3a-4781-a063-ee817fc2fb1b admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:26.956708 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-32697ceb-af94-480e-8fa1-a834476882c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f5f939a10574c1a9614e3cdd6432a62'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.966604 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-32697ceb-af94-480e-8fa1-a834476882c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f5f939a10574c1a9614e3cdd6432a62'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.967589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-32697ceb-af94-480e-8fa1-a834476882c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f5f939a10574c1a9614e3cdd6432a62'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.975376 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5b6a8877-af3a-4781-a063-ee817fc2fb1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43e3ed6833064b3e8cf38caa0e7d1cf2', 'request_id': 'req-5b6a8877-af3a-4781-a063-ee817fc2fb1b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.981823 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5b6a8877-af3a-4781-a063-ee817fc2fb1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43e3ed6833064b3e8cf38caa0e7d1cf2', 'request_id': 'req-5b6a8877-af3a-4781-a063-ee817fc2fb1b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.982934 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-32697ceb-af94-480e-8fa1-a834476882c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f5f939a10574c1a9614e3cdd6432a62'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.992919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5b6a8877-af3a-4781-a063-ee817fc2fb1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43e3ed6833064b3e8cf38caa0e7d1cf2', 'request_id': 'req-5b6a8877-af3a-4781-a063-ee817fc2fb1b'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:26.997067 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-32697ceb-af94-480e-8fa1-a834476882c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f5f939a10574c1a9614e3cdd6432a62'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.004199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1368/2703] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:26 2022] DELETE /identity/v3/users/1f5f939a10574c1a9614e3cdd6432a62 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:27.007766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5b6a8877-af3a-4781-a063-ee817fc2fb1b admin admin] The token cache is being invalidate because project 43e3ed6833064b3e8cf38caa0e7d1cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:27.008169 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5b6a8877-af3a-4781-a063-ee817fc2fb1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.008818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5b6a8877-af3a-4781-a063-ee817fc2fb1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.011105 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1336/2704] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:26 2022] DELETE /identity/v3/projects/43e3ed6833064b3e8cf38caa0e7d1cf2 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:27.012027 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966b349-ab0a-4ed5-b532-e2c2d060a989 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.012579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966b349-ab0a-4ed5-b532-e2c2d060a989 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.013155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966b349-ab0a-4ed5-b532-e2c2d060a989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.016604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbb56a-5e7a-4067-91f7-b6f94ff59557 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.016887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbb56a-5e7a-4067-91f7-b6f94ff59557 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.017159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbb56a-5e7a-4067-91f7-b6f94ff59557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.038728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-71bbb56a-5e7a-4067-91f7-b6f94ff59557 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:27.039980 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-71bbb56a-5e7a-4067-91f7-b6f94ff59557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:27.040622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4966b349-ab0a-4ed5-b532-e2c2d060a989 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:27.041914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4966b349-ab0a-4ed5-b532-e2c2d060a989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:27.086525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:27.119625 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1369/2705] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:27.120870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1337/2706] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:27.130070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966b349-ab0a-4ed5-b532-e2c2d060a989 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:27.131933 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4966b349-ab0a-4ed5-b532-e2c2d060a989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:27.185919 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54965b85-bb7c-4d96-b92b-ee67999dde5a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.186259 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54965b85-bb7c-4d96-b92b-ee67999dde5a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.186479 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54965b85-bb7c-4d96-b92b-ee67999dde5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.191801 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:27.209329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-54965b85-bb7c-4d96-b92b-ee67999dde5a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:27.210550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-54965b85-bb7c-4d96-b92b-ee67999dde5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:27.216207 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966b349-ab0a-4ed5-b532-e2c2d060a989 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:27.218783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1338/2707] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:27.220873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4230f-1f30-403e-a8c6-5a7c55ee0be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:27.221603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4230f-1f30-403e-a8c6-5a7c55ee0be1 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.222025 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4230f-1f30-403e-a8c6-5a7c55ee0be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.222306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4230f-1f30-403e-a8c6-5a7c55ee0be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.224518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54965b85-bb7c-4d96-b92b-ee67999dde5a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:27.234322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4230f-1f30-403e-a8c6-5a7c55ee0be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:27.236010 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4230f-1f30-403e-a8c6-5a7c55ee0be1 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:27.236889 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54965b85-bb7c-4d96-b92b-ee67999dde5a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:27.245633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c116d3d-14a8-48f3-87e0-7ce6f245e63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:27.246454 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c116d3d-14a8-48f3-87e0-7ce6f245e63e admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.246865 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c116d3d-14a8-48f3-87e0-7ce6f245e63e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.247279 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c116d3d-14a8-48f3-87e0-7ce6f245e63e admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.247643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1370/2708] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:27.248232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c116d3d-14a8-48f3-87e0-7ce6f245e63e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:27.248551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c116d3d-14a8-48f3-87e0-7ce6f245e63e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:27.250542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c116d3d-14a8-48f3-87e0-7ce6f245e63e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:27.265492 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1339/2709] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:59:27.270427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4230f-1f30-403e-a8c6-5a7c55ee0be1 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:27.280521 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4230f-1f30-403e-a8c6-5a7c55ee0be1 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:27.286263 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4f133-65dc-4928-856c-9bd67a903f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:27.287049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4f133-65dc-4928-856c-9bd67a903f03 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.287516 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4f133-65dc-4928-856c-9bd67a903f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.287903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4f133-65dc-4928-856c-9bd67a903f03 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.288569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4f133-65dc-4928-856c-9bd67a903f03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:27.290840 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4f133-65dc-4928-856c-9bd67a903f03 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:27.321070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1371/2710] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:27 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:27.325486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c116d3d-14a8-48f3-87e0-7ce6f245e63e admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:27.348534 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c116d3d-14a8-48f3-87e0-7ce6f245e63e admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:27.435316 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b0573-785b-4c84-9d38-315787070869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:27.435968 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b0573-785b-4c84-9d38-315787070869 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.436248 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b0573-785b-4c84-9d38-315787070869 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.439478 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b0573-785b-4c84-9d38-315787070869 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.439478 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b0573-785b-4c84-9d38-315787070869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:27.439478 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b0573-785b-4c84-9d38-315787070869 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:27.465740 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1340/2711] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:27 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:27.469536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4f133-65dc-4928-856c-9bd67a903f03 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:27.482583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4f133-65dc-4928-856c-9bd67a903f03 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:27.492371 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742f97c-92f0-4e19-b43e-ab10b87e4781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:27.493194 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742f97c-92f0-4e19-b43e-ab10b87e4781 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.493718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742f97c-92f0-4e19-b43e-ab10b87e4781 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.494168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742f97c-92f0-4e19-b43e-ab10b87e4781 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.495274 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742f97c-92f0-4e19-b43e-ab10b87e4781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:27.497442 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742f97c-92f0-4e19-b43e-ab10b87e4781 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:27.511724 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1372/2712] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:27.514026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b0573-785b-4c84-9d38-315787070869 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:27.525215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b0573-785b-4c84-9d38-315787070869 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:27.529914 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bc52f-9452-409c-bd2f-b8f467d082db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:27.530516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bc52f-9452-409c-bd2f-b8f467d082db admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.530795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bc52f-9452-409c-bd2f-b8f467d082db admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.531069 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bc52f-9452-409c-bd2f-b8f467d082db admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.531764 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bc52f-9452-409c-bd2f-b8f467d082db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:27.533347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bc52f-9452-409c-bd2f-b8f467d082db admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:27.547497 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1341/2713] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:27.548446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742f97c-92f0-4e19-b43e-ab10b87e4781 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:27.559463 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742f97c-92f0-4e19-b43e-ab10b87e4781 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:27.568446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5240f-1741-4db2-ace0-0d9ac2f35099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:27.569340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5240f-1741-4db2-ace0-0d9ac2f35099 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.569504 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5240f-1741-4db2-ace0-0d9ac2f35099 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.569833 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5240f-1741-4db2-ace0-0d9ac2f35099 admin admin] PATH_INFO: `/v3/projects/56222de93bf841ed82f6683c1b1f1cf9/users/807ee9adf7d44276829cf53e531151af/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.586237 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bc52f-9452-409c-bd2f-b8f467d082db admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:27.588324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ac7bc52f-9452-409c-bd2f-b8f467d082db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:27.591877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5240f-1741-4db2-ace0-0d9ac2f35099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56222de93bf841ed82f6683c1b1f1cf9, user_id=807ee9adf7d44276829cf53e531151af, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:27.594918 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5240f-1741-4db2-ace0-0d9ac2f35099 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:27.605854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bc52f-9452-409c-bd2f-b8f467d082db admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:27.611839 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:27.612478 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.612815 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.613076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] PATH_INFO: `/v3/projects/19c522cfb1cf4d20a8579c13fa581059` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.614496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19c522cfb1cf4d20a8579c13fa581059)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:27.616431 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:27.621262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1373/2714] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:27 2022] PUT /identity/v3/projects/56222de93bf841ed82f6683c1b1f1cf9/users/807ee9adf7d44276829cf53e531151af/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:27.631576 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b005d-177e-43d8-b5f9-810381215bce None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.631881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b005d-177e-43d8-b5f9-810381215bce None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.632155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b005d-177e-43d8-b5f9-810381215bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.661510 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-c07b005d-177e-43d8-b5f9-810381215bce None None] MFA Rules not processed for user `807ee9adf7d44276829cf53e531151af`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:27.662725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c07b005d-177e-43d8-b5f9-810381215bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:27.666014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19c522cfb1cf4d20a8579c13fa581059', 'request_id': 'req-42815809-158c-4518-af52-7220af18e1c3'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.671546 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19c522cfb1cf4d20a8579c13fa581059', 'request_id': 'req-42815809-158c-4518-af52-7220af18e1c3'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.680783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19c522cfb1cf4d20a8579c13fa581059', 'request_id': 'req-42815809-158c-4518-af52-7220af18e1c3'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.692149 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] The token cache is being invalidate because project 19c522cfb1cf4d20a8579c13fa581059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:27.692435 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.693082 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.694566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1342/2715] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:27 2022] DELETE /identity/v3/projects/19c522cfb1cf4d20a8579c13fa581059 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:27.698433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:27.700347 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:27.727713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:27.737289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1374/2716] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:27.748856 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b005d-177e-43d8-b5f9-810381215bce None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:27.750251 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c07b005d-177e-43d8-b5f9-810381215bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:27.771046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42815809-158c-4518-af52-7220af18e1c3 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:27.776001 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acda8fa-038c-4eda-8bc4-a23c044bd2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:27.776659 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acda8fa-038c-4eda-8bc4-a23c044bd2e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.777034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acda8fa-038c-4eda-8bc4-a23c044bd2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.777408 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acda8fa-038c-4eda-8bc4-a23c044bd2e5 admin admin] PATH_INFO: `/v3/users/4094a747725d4078b651a7cbe0e666ae` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.778537 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acda8fa-038c-4eda-8bc4-a23c044bd2e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4094a747725d4078b651a7cbe0e666ae)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:27.780565 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acda8fa-038c-4eda-8bc4-a23c044bd2e5 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:27.811616 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b005d-177e-43d8-b5f9-810381215bce None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:27.815485 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e324b1f-616d-4708-8681-278d2c34e9a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:27.816126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e324b1f-616d-4708-8681-278d2c34e9a5 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:27.816421 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e324b1f-616d-4708-8681-278d2c34e9a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:27.816698 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e324b1f-616d-4708-8681-278d2c34e9a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:27.825395 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e324b1f-616d-4708-8681-278d2c34e9a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:27.826299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4acda8fa-038c-4eda-8bc4-a23c044bd2e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4094a747725d4078b651a7cbe0e666ae'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.826818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e324b1f-616d-4708-8681-278d2c34e9a5 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:27.833798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4acda8fa-038c-4eda-8bc4-a23c044bd2e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4094a747725d4078b651a7cbe0e666ae'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.834500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4acda8fa-038c-4eda-8bc4-a23c044bd2e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4094a747725d4078b651a7cbe0e666ae'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.835793 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4acda8fa-038c-4eda-8bc4-a23c044bd2e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4094a747725d4078b651a7cbe0e666ae'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.837037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1375/2717] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:59:27.848851 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-4acda8fa-038c-4eda-8bc4-a23c044bd2e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4094a747725d4078b651a7cbe0e666ae'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:27.863243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1343/2718] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:27 2022] DELETE /identity/v3/users/4094a747725d4078b651a7cbe0e666ae => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:28.164652 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e324b1f-616d-4708-8681-278d2c34e9a5 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:28.197501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e324b1f-616d-4708-8681-278d2c34e9a5 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:28.211018 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:28.256225 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa277e6-3795-4b29-9e8b-9a7f13831e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:28.256881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa277e6-3795-4b29-9e8b-9a7f13831e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:28.257161 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa277e6-3795-4b29-9e8b-9a7f13831e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:28.257434 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa277e6-3795-4b29-9e8b-9a7f13831e03 admin admin] PATH_INFO: `/v3/projects/ab0c871d41294ac4ba92df0913346fef` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:28.258533 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa277e6-3795-4b29-9e8b-9a7f13831e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0c871d41294ac4ba92df0913346fef)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:28.260587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa277e6-3795-4b29-9e8b-9a7f13831e03 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:28.307754 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-aaa277e6-3795-4b29-9e8b-9a7f13831e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0c871d41294ac4ba92df0913346fef', 'request_id': 'req-aaa277e6-3795-4b29-9e8b-9a7f13831e03'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:28.313034 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-aaa277e6-3795-4b29-9e8b-9a7f13831e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0c871d41294ac4ba92df0913346fef', 'request_id': 'req-aaa277e6-3795-4b29-9e8b-9a7f13831e03'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:28.319629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-aaa277e6-3795-4b29-9e8b-9a7f13831e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0c871d41294ac4ba92df0913346fef', 'request_id': 'req-aaa277e6-3795-4b29-9e8b-9a7f13831e03'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:28.330382 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-aaa277e6-3795-4b29-9e8b-9a7f13831e03 admin admin] The token cache is being invalidate because project ab0c871d41294ac4ba92df0913346fef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:28.330666 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-aaa277e6-3795-4b29-9e8b-9a7f13831e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:28.331268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-aaa277e6-3795-4b29-9e8b-9a7f13831e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:28.332749 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1376/2719] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:28 2022] DELETE /identity/v3/projects/ab0c871d41294ac4ba92df0913346fef => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:28.337588 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44899e1f-074b-4699-9f53-99ad945e2454 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:28.338068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44899e1f-074b-4699-9f53-99ad945e2454 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:28.338181 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44899e1f-074b-4699-9f53-99ad945e2454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:28.357688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-44899e1f-074b-4699-9f53-99ad945e2454 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:28.358896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-44899e1f-074b-4699-9f53-99ad945e2454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:28.418024 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1344/2720] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:28.425812 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38643c-6675-44fb-a229-96c42b32fb3a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:28.426101 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38643c-6675-44fb-a229-96c42b32fb3a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:28.426375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38643c-6675-44fb-a229-96c42b32fb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:28.446327 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3f38643c-6675-44fb-a229-96c42b32fb3a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:28.447541 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3f38643c-6675-44fb-a229-96c42b32fb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:28.456175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1377/2721] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:28.464883 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44899e1f-074b-4699-9f53-99ad945e2454 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:28.470825 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44899e1f-074b-4699-9f53-99ad945e2454 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:28.475597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92e634-c86e-453a-a3d9-cd0cc706427b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:28.476231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92e634-c86e-453a-a3d9-cd0cc706427b admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:28.476510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92e634-c86e-453a-a3d9-cd0cc706427b admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:28.476784 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92e634-c86e-453a-a3d9-cd0cc706427b admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:28.477455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92e634-c86e-453a-a3d9-cd0cc706427b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:28.477762 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92e634-c86e-453a-a3d9-cd0cc706427b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:28.479357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92e634-c86e-453a-a3d9-cd0cc706427b admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:28.492733 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1345/2722] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:59:28.497619 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38643c-6675-44fb-a229-96c42b32fb3a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:28.507638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38643c-6675-44fb-a229-96c42b32fb3a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:28.512312 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8365f3a-8f09-438b-946e-475d0462b943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:28.512905 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8365f3a-8f09-438b-946e-475d0462b943 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:28.513183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8365f3a-8f09-438b-946e-475d0462b943 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:28.513451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8365f3a-8f09-438b-946e-475d0462b943 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:28.514313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8365f3a-8f09-438b-946e-475d0462b943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:28.516250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8365f3a-8f09-438b-946e-475d0462b943 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:28.546205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1378/2723] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:28 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:28.550430 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92e634-c86e-453a-a3d9-cd0cc706427b admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:28.560407 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92e634-c86e-453a-a3d9-cd0cc706427b admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:28.587238 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:28.621551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e04944-cc0f-4ec8-83de-4fe638197623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:28.622151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e04944-cc0f-4ec8-83de-4fe638197623 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:28.622427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e04944-cc0f-4ec8-83de-4fe638197623 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:28.622692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e04944-cc0f-4ec8-83de-4fe638197623 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:28.623306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e04944-cc0f-4ec8-83de-4fe638197623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:28.625279 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e04944-cc0f-4ec8-83de-4fe638197623 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:28.646802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1346/2724] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:28 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:28.651157 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8365f3a-8f09-438b-946e-475d0462b943 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:28.661987 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8365f3a-8f09-438b-946e-475d0462b943 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:28.666777 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b24f1f-946e-4ad2-9e81-ee97a983736b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:28.667373 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b24f1f-946e-4ad2-9e81-ee97a983736b admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:28.667651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b24f1f-946e-4ad2-9e81-ee97a983736b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:28.667964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b24f1f-946e-4ad2-9e81-ee97a983736b admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:28.668602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b24f1f-946e-4ad2-9e81-ee97a983736b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:28.670218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b24f1f-946e-4ad2-9e81-ee97a983736b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:28.680325 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1379/2725] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:28.684178 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e04944-cc0f-4ec8-83de-4fe638197623 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:28.695486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e04944-cc0f-4ec8-83de-4fe638197623 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:28.700616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac54c7f-d18e-47be-a15b-939884750e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:28.701215 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac54c7f-d18e-47be-a15b-939884750e27 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:28.701542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac54c7f-d18e-47be-a15b-939884750e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:28.701804 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac54c7f-d18e-47be-a15b-939884750e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:28.702460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac54c7f-d18e-47be-a15b-939884750e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:28.704077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac54c7f-d18e-47be-a15b-939884750e27 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:28.714469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1347/2726] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:28.718579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b24f1f-946e-4ad2-9e81-ee97a983736b admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:28.729078 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b24f1f-946e-4ad2-9e81-ee97a983736b admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:28.733721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15343a19-55ad-48ba-bbc9-ea17ae59cc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:28.734360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15343a19-55ad-48ba-bbc9-ea17ae59cc82 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:28.734641 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15343a19-55ad-48ba-bbc9-ea17ae59cc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:28.734914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15343a19-55ad-48ba-bbc9-ea17ae59cc82 admin admin] PATH_INFO: `/v3/projects/a2af625f97834e43a72f7ec73f629514/users/2da4b64e3fee42718c5f170f82207fca/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:28.749183 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15343a19-55ad-48ba-bbc9-ea17ae59cc82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2af625f97834e43a72f7ec73f629514, user_id=2da4b64e3fee42718c5f170f82207fca, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:28.751847 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15343a19-55ad-48ba-bbc9-ea17ae59cc82 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:28.765583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1380/2727] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:28 2022] PUT /identity/v3/projects/a2af625f97834e43a72f7ec73f629514/users/2da4b64e3fee42718c5f170f82207fca/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:28.776447 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de67e6e-d740-48fe-a913-f8f98db17f59 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:28.776755 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de67e6e-d740-48fe-a913-f8f98db17f59 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:28.777050 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de67e6e-d740-48fe-a913-f8f98db17f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:28.804783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-6de67e6e-d740-48fe-a913-f8f98db17f59 None None] MFA Rules not processed for user `2da4b64e3fee42718c5f170f82207fca`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:28.805993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6de67e6e-d740-48fe-a913-f8f98db17f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:28.868885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1348/2728] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:28.880174 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15343a19-55ad-48ba-bbc9-ea17ae59cc82 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:28.881860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-15343a19-55ad-48ba-bbc9-ea17ae59cc82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:28.929130 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:28.942136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15343a19-55ad-48ba-bbc9-ea17ae59cc82 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:28.945956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b2975-5cb4-4549-9bda-11dfcf1306a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f2b6dc3406f46e68c74677f5fc7e3b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:28.946600 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b2975-5cb4-4549-9bda-11dfcf1306a3 service glance] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:28.946873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b2975-5cb4-4549-9bda-11dfcf1306a3 service glance] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:28.947143 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b2975-5cb4-4549-9bda-11dfcf1306a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:28.955298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b2975-5cb4-4549-9bda-11dfcf1306a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:28.956752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b2975-5cb4-4549-9bda-11dfcf1306a3 service glance] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:28.968041 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1381/2729] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:37.095652 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de67e6e-d740-48fe-a913-f8f98db17f59 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:37.097204 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6de67e6e-d740-48fe-a913-f8f98db17f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:37.197464 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de67e6e-d740-48fe-a913-f8f98db17f59 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:37.202516 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ab33a-bb4a-43dd-87d0-06f6aed34cd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:37.203118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ab33a-bb4a-43dd-87d0-06f6aed34cd7 service placement] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:37.203396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ab33a-bb4a-43dd-87d0-06f6aed34cd7 service placement] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:37.203667 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ab33a-bb4a-43dd-87d0-06f6aed34cd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:37.205413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-762ab33a-bb4a-43dd-87d0-06f6aed34cd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:37.218374 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ab33a-bb4a-43dd-87d0-06f6aed34cd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:37.220054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ab33a-bb4a-43dd-87d0-06f6aed34cd7 service placement] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:37.231113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1349/2730] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:38.963088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=6 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:43.792178 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f9836-06b3-4fa1-b68d-cfa6eafe99bf None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:43.792782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f9836-06b3-4fa1-b68d-cfa6eafe99bf None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:43.792782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f9836-06b3-4fa1-b68d-cfa6eafe99bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:43.812095 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-8c1f9836-06b3-4fa1-b68d-cfa6eafe99bf None None] MFA Rules not processed for user `ac4d918f549c41d3ae19e80cecefa95c`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:43.813389 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-8c1f9836-06b3-4fa1-b68d-cfa6eafe99bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:43.871394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1382/2731] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:44.425774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ab33a-bb4a-43dd-87d0-06f6aed34cd7 service placement] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:44.427538 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-762ab33a-bb4a-43dd-87d0-06f6aed34cd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:44.439227 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:44.505224 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ab33a-bb4a-43dd-87d0-06f6aed34cd7 service placement] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:44.512101 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:44.512834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:44.513189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:44.513607 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] PATH_INFO: `/v3/users/009059267bbf430c9d592bc9b1271e51` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:44.515559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=009059267bbf430c9d592bc9b1271e51)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:44.517753 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:44.578101 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '009059267bbf430c9d592bc9b1271e51'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:44.594894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '009059267bbf430c9d592bc9b1271e51'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:44.594894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '009059267bbf430c9d592bc9b1271e51'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:44.594894 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '009059267bbf430c9d592bc9b1271e51'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:44.608044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '009059267bbf430c9d592bc9b1271e51'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:44.625236 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1350/2732] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:44 2022] DELETE /identity/v3/users/009059267bbf430c9d592bc9b1271e51 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:44.629867 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40c90d-d4d3-4de1-8cd6-e5f66c67e270 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:44.630346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40c90d-d4d3-4de1-8cd6-e5f66c67e270 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:44.630747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40c90d-d4d3-4de1-8cd6-e5f66c67e270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:44.668771 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-9a40c90d-d4d3-4de1-8cd6-e5f66c67e270 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:44.670112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9a40c90d-d4d3-4de1-8cd6-e5f66c67e270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:44.760499 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1383/2733] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:44.770958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:44.772494 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:44.870790 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864ac72-1fce-48f7-a67d-53cbdfac5476 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:44.876176 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bb952-9ac9-4f73-8f93-5f2087120ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:44.877021 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bb952-9ac9-4f73-8f93-5f2087120ecb service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:44.877475 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bb952-9ac9-4f73-8f93-5f2087120ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:44.877918 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bb952-9ac9-4f73-8f93-5f2087120ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:44.888236 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5bb952-9ac9-4f73-8f93-5f2087120ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:44.890035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5bb952-9ac9-4f73-8f93-5f2087120ecb service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:44.901061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1351/2734] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:59:45.197301 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40c90d-d4d3-4de1-8cd6-e5f66c67e270 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:45.198910 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-9a40c90d-d4d3-4de1-8cd6-e5f66c67e270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:45.221399 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40c90d-d4d3-4de1-8cd6-e5f66c67e270 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:45.228112 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4498f4b-8345-49b3-97bb-18fe278de767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:45.228927 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4498f4b-8345-49b3-97bb-18fe278de767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:45.229304 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4498f4b-8345-49b3-97bb-18fe278de767 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:45.229644 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4498f4b-8345-49b3-97bb-18fe278de767 admin admin] PATH_INFO: `/v3/projects/622090417eb3426fbbf971444b821e75` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:45.231733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4498f4b-8345-49b3-97bb-18fe278de767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=622090417eb3426fbbf971444b821e75)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:45.233865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4498f4b-8345-49b3-97bb-18fe278de767 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:45.294817 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4498f4b-8345-49b3-97bb-18fe278de767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '622090417eb3426fbbf971444b821e75', 'request_id': 'req-d4498f4b-8345-49b3-97bb-18fe278de767'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:45.300346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4498f4b-8345-49b3-97bb-18fe278de767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '622090417eb3426fbbf971444b821e75', 'request_id': 'req-d4498f4b-8345-49b3-97bb-18fe278de767'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:45.308040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4498f4b-8345-49b3-97bb-18fe278de767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '622090417eb3426fbbf971444b821e75', 'request_id': 'req-d4498f4b-8345-49b3-97bb-18fe278de767'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:45.319365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4498f4b-8345-49b3-97bb-18fe278de767 admin admin] The token cache is being invalidate because project 622090417eb3426fbbf971444b821e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:45.319654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4498f4b-8345-49b3-97bb-18fe278de767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:45.320296 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-d4498f4b-8345-49b3-97bb-18fe278de767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:45.321736 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1384/2735] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:45 2022] DELETE /identity/v3/projects/622090417eb3426fbbf971444b821e75 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:45.328125 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268d25a-459f-4d31-8f31-17887e416208 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:45.328404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268d25a-459f-4d31-8f31-17887e416208 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:45.328671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268d25a-459f-4d31-8f31-17887e416208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:45.348869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-7268d25a-459f-4d31-8f31-17887e416208 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:45.350472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7268d25a-459f-4d31-8f31-17887e416208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:45.355557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:45.415501 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1352/2736] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:45.423358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab17ca-1ec1-4752-b37b-3d1cb257d033 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:45.423636 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab17ca-1ec1-4752-b37b-3d1cb257d033 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:45.423939 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab17ca-1ec1-4752-b37b-3d1cb257d033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:45.443402 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-a4ab17ca-1ec1-4752-b37b-3d1cb257d033 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:45.444571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a4ab17ca-1ec1-4752-b37b-3d1cb257d033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:45.452358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1385/2737] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:45.456675 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268d25a-459f-4d31-8f31-17887e416208 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:45.466698 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268d25a-459f-4d31-8f31-17887e416208 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:45.471268 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96daf955-9cdc-48d3-ad2e-8f0ea5371330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:45.471886 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daf955-9cdc-48d3-ad2e-8f0ea5371330 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:45.472164 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daf955-9cdc-48d3-ad2e-8f0ea5371330 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:45.472434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daf955-9cdc-48d3-ad2e-8f0ea5371330 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:45.473070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96daf955-9cdc-48d3-ad2e-8f0ea5371330 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:45.473369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96daf955-9cdc-48d3-ad2e-8f0ea5371330 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:45.474958 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96daf955-9cdc-48d3-ad2e-8f0ea5371330 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:45.487097 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1353/2738] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:59:45.492446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab17ca-1ec1-4752-b37b-3d1cb257d033 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:45.501816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab17ca-1ec1-4752-b37b-3d1cb257d033 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:45.506742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca955e35-2f9c-4f7e-87c2-db3d092db677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:45.507355 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca955e35-2f9c-4f7e-87c2-db3d092db677 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:45.507628 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca955e35-2f9c-4f7e-87c2-db3d092db677 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:45.508271 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca955e35-2f9c-4f7e-87c2-db3d092db677 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:45.508899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca955e35-2f9c-4f7e-87c2-db3d092db677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:45.510814 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca955e35-2f9c-4f7e-87c2-db3d092db677 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:45.533705 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:45.542390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1386/2739] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:45 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:45.546434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96daf955-9cdc-48d3-ad2e-8f0ea5371330 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:45.556318 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96daf955-9cdc-48d3-ad2e-8f0ea5371330 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:45.611725 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd8381-8248-41b0-8289-4c1a1d98c726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:45.612331 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd8381-8248-41b0-8289-4c1a1d98c726 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:45.612611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd8381-8248-41b0-8289-4c1a1d98c726 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:45.612885 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd8381-8248-41b0-8289-4c1a1d98c726 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:45.613511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd8381-8248-41b0-8289-4c1a1d98c726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:45.615455 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd8381-8248-41b0-8289-4c1a1d98c726 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:45.636729 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1354/2740] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:45 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:59:45.641385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca955e35-2f9c-4f7e-87c2-db3d092db677 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:45.651915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca955e35-2f9c-4f7e-87c2-db3d092db677 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:45.656691 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544fd1d-ca06-467a-9eb9-d2345668f3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:45.657283 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544fd1d-ca06-467a-9eb9-d2345668f3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:45.657555 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544fd1d-ca06-467a-9eb9-d2345668f3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:45.657822 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544fd1d-ca06-467a-9eb9-d2345668f3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:45.658454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544fd1d-ca06-467a-9eb9-d2345668f3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:45.660094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544fd1d-ca06-467a-9eb9-d2345668f3e7 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:45.670490 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1387/2741] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:45.674276 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd8381-8248-41b0-8289-4c1a1d98c726 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:45.685327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd8381-8248-41b0-8289-4c1a1d98c726 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:45.690694 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784239b-69d7-41c2-b199-a4795a292768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:45.690859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784239b-69d7-41c2-b199-a4795a292768 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:45.691107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784239b-69d7-41c2-b199-a4795a292768 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:45.691383 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784239b-69d7-41c2-b199-a4795a292768 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:45.692087 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784239b-69d7-41c2-b199-a4795a292768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:45.693731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784239b-69d7-41c2-b199-a4795a292768 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:45.708177 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1355/2742] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:45.712578 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544fd1d-ca06-467a-9eb9-d2345668f3e7 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:45.724292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544fd1d-ca06-467a-9eb9-d2345668f3e7 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:45.730218 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6e052-cc35-4a8d-9605-4e9e5afc4c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:45.730884 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6e052-cc35-4a8d-9605-4e9e5afc4c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:45.731365 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6e052-cc35-4a8d-9605-4e9e5afc4c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:45.731606 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6e052-cc35-4a8d-9605-4e9e5afc4c6a admin admin] PATH_INFO: `/v3/projects/379b08d3cf834ca9b891abd6ba0f14ee/users/1e3a4f725f3d41edbb7c162a8760bf8b/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:45.750465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6e052-cc35-4a8d-9605-4e9e5afc4c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=379b08d3cf834ca9b891abd6ba0f14ee, user_id=1e3a4f725f3d41edbb7c162a8760bf8b, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:45.753549 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6e052-cc35-4a8d-9605-4e9e5afc4c6a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:45.768412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1388/2743] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:45 2022] PUT /identity/v3/projects/379b08d3cf834ca9b891abd6ba0f14ee/users/1e3a4f725f3d41edbb7c162a8760bf8b/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:45.773865 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400e7c9-4213-4e02-8c71-2e678d356d7d None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:45.774154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400e7c9-4213-4e02-8c71-2e678d356d7d None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:45.774426 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400e7c9-4213-4e02-8c71-2e678d356d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:45.794493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-7400e7c9-4213-4e02-8c71-2e678d356d7d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:45.795804 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7400e7c9-4213-4e02-8c71-2e678d356d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:45.870390 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1356/2744] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:45.881186 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6e052-cc35-4a8d-9605-4e9e5afc4c6a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:45.882650 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b0c6e052-cc35-4a8d-9605-4e9e5afc4c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:46.000430 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6e052-cc35-4a8d-9605-4e9e5afc4c6a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:46.005502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa10b39-20dd-4cc6-a481-1c18337689e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:46.006141 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa10b39-20dd-4cc6-a481-1c18337689e6 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:46.006420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa10b39-20dd-4cc6-a481-1c18337689e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:46.006722 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa10b39-20dd-4cc6-a481-1c18337689e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:46.018092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa10b39-20dd-4cc6-a481-1c18337689e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:46.019662 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa10b39-20dd-4cc6-a481-1c18337689e6 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:46.030450 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1389/2745] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:50.973094 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d2ffd-615f-4011-a8f2-e426e68cd8ab None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:50.975229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d2ffd-615f-4011-a8f2-e426e68cd8ab None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:50.976294 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d2ffd-615f-4011-a8f2-e426e68cd8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:51.018525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-a50d2ffd-615f-4011-a8f2-e426e68cd8ab None None] MFA Rules not processed for user `1e3a4f725f3d41edbb7c162a8760bf8b`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:51.020550 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a50d2ffd-615f-4011-a8f2-e426e68cd8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:51.025197 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:51.111887 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1357/2746] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:51.117102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa10b39-20dd-4cc6-a481-1c18337689e6 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:51.128677 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa10b39-20dd-4cc6-a481-1c18337689e6 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:51.134384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d7438-0c73-4753-811d-0e87c0a68c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:51.135270 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d7438-0c73-4753-811d-0e87c0a68c44 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:51.135750 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d7438-0c73-4753-811d-0e87c0a68c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:51.136197 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d7438-0c73-4753-811d-0e87c0a68c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:51.137058 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d7438-0c73-4753-811d-0e87c0a68c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:51.139272 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d7438-0c73-4753-811d-0e87c0a68c44 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:51.191745 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1390/2747] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:51 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:51.196493 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d2ffd-615f-4011-a8f2-e426e68cd8ab None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:51.208113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d2ffd-615f-4011-a8f2-e426e68cd8ab None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:51.270481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4f428-2fa3-47da-bf0e-5a43f9010e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:51.271007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4f428-2fa3-47da-bf0e-5a43f9010e8c admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:51.271290 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4f428-2fa3-47da-bf0e-5a43f9010e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:51.271566 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4f428-2fa3-47da-bf0e-5a43f9010e8c admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:51.272257 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4f428-2fa3-47da-bf0e-5a43f9010e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:51.274946 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4f428-2fa3-47da-bf0e-5a43f9010e8c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:51.296369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1358/2748] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:51 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:51.300637 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d7438-0c73-4753-811d-0e87c0a68c44 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:51.311108 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d7438-0c73-4753-811d-0e87c0a68c44 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:51.315623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba91f2-d163-4aed-8679-45ad385320df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:51.316236 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba91f2-d163-4aed-8679-45ad385320df admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:51.316513 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba91f2-d163-4aed-8679-45ad385320df admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:51.316782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba91f2-d163-4aed-8679-45ad385320df admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:51.317416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba91f2-d163-4aed-8679-45ad385320df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:51.319008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba91f2-d163-4aed-8679-45ad385320df admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:51.330931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1391/2749] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:51.334938 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4f428-2fa3-47da-bf0e-5a43f9010e8c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:51.346002 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4f428-2fa3-47da-bf0e-5a43f9010e8c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:51.350860 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131dd7b-3e4d-4566-a624-11ca15dd68d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:51.351525 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131dd7b-3e4d-4566-a624-11ca15dd68d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:51.351834 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131dd7b-3e4d-4566-a624-11ca15dd68d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:51.352109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131dd7b-3e4d-4566-a624-11ca15dd68d3 admin admin] PATH_INFO: `/v3/projects/80b2b237be46413fa8742f0662512149/users/e105e9d96c7d4cc296745ee791efaa2e/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:51.367521 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131dd7b-3e4d-4566-a624-11ca15dd68d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b2b237be46413fa8742f0662512149, user_id=e105e9d96c7d4cc296745ee791efaa2e, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:51.370191 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131dd7b-3e4d-4566-a624-11ca15dd68d3 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:51.383232 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1359/2750] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:51 2022] PUT /identity/v3/projects/80b2b237be46413fa8742f0662512149/users/e105e9d96c7d4cc296745ee791efaa2e/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:51.388837 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba91f2-d163-4aed-8679-45ad385320df admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:51.401064 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba91f2-d163-4aed-8679-45ad385320df admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:51.459026 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d19550-f6b7-4f0b-8bf6-20d4ad203d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:51.459742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d19550-f6b7-4f0b-8bf6-20d4ad203d78 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:51.460720 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d19550-f6b7-4f0b-8bf6-20d4ad203d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:51.461002 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d19550-f6b7-4f0b-8bf6-20d4ad203d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:51.461635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d19550-f6b7-4f0b-8bf6-20d4ad203d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:51.463215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d19550-f6b7-4f0b-8bf6-20d4ad203d78 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:51.473090 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1392/2751] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:51.479229 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131dd7b-3e4d-4566-a624-11ca15dd68d3 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:51.489631 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131dd7b-3e4d-4566-a624-11ca15dd68d3 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:51.495620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af295e-8b30-42c6-9d3f-12ab60939ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:51.496325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af295e-8b30-42c6-9d3f-12ab60939ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:51.496595 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af295e-8b30-42c6-9d3f-12ab60939ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:51.496890 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af295e-8b30-42c6-9d3f-12ab60939ee8 admin admin] PATH_INFO: `/v3/projects/80b2b237be46413fa8742f0662512149/users/e105e9d96c7d4cc296745ee791efaa2e/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:51.499012 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af295e-8b30-42c6-9d3f-12ab60939ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b2b237be46413fa8742f0662512149, user_id=e105e9d96c7d4cc296745ee791efaa2e, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:51.502109 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af295e-8b30-42c6-9d3f-12ab60939ee8 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:51.515073 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1360/2752] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:51 2022] PUT /identity/v3/projects/80b2b237be46413fa8742f0662512149/users/e105e9d96c7d4cc296745ee791efaa2e/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:53.305016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d19550-f6b7-4f0b-8bf6-20d4ad203d78 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:53.306934 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-71d19550-f6b7-4f0b-8bf6-20d4ad203d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:53.320686 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d19550-f6b7-4f0b-8bf6-20d4ad203d78 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:53.355264 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:53.396906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:53.398124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:53.398424 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:53.399476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] PATH_INFO: `/v3/users/2da4b64e3fee42718c5f170f82207fca` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:53.400943 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da4b64e3fee42718c5f170f82207fca)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:53.403756 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:53.510039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da4b64e3fee42718c5f170f82207fca'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:53.523556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da4b64e3fee42718c5f170f82207fca'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:53.524564 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da4b64e3fee42718c5f170f82207fca'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:53.542855 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da4b64e3fee42718c5f170f82207fca'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:53.557635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da4b64e3fee42718c5f170f82207fca'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:53.571723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1393/2753] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:53 2022] DELETE /identity/v3/users/2da4b64e3fee42718c5f170f82207fca => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:53.576144 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f274b-a6f5-4f7d-b5bd-68444bc58a47 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:53.577010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f274b-a6f5-4f7d-b5bd-68444bc58a47 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:53.577330 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f274b-a6f5-4f7d-b5bd-68444bc58a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:53.605032 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-777f274b-a6f5-4f7d-b5bd-68444bc58a47 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:53.606599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-777f274b-a6f5-4f7d-b5bd-68444bc58a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:53.692761 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:53.710859 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1361/2754] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:53.723385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:53.725243 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:53.809290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78f5ec-13e0-4456-8ecc-09c47acda831 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:53.814181 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4e113-f554-4e85-bd1e-645d7cb31b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:53.815077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4e113-f554-4e85-bd1e-645d7cb31b7f service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:53.815483 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4e113-f554-4e85-bd1e-645d7cb31b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:53.815906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4e113-f554-4e85-bd1e-645d7cb31b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:53.832454 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4e113-f554-4e85-bd1e-645d7cb31b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:53.834076 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4e113-f554-4e85-bd1e-645d7cb31b7f service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:53.862104 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1394/2755] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:54.330157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f274b-a6f5-4f7d-b5bd-68444bc58a47 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:54.331937 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-777f274b-a6f5-4f7d-b5bd-68444bc58a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:54.375822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f274b-a6f5-4f7d-b5bd-68444bc58a47 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:54.387800 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:54.388556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:54.388847 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:54.389131 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] PATH_INFO: `/v3/projects/a2af625f97834e43a72f7ec73f629514` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:54.390780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2af625f97834e43a72f7ec73f629514)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:54.392913 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:54.501515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2af625f97834e43a72f7ec73f629514', 'request_id': 'req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:54.521721 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2af625f97834e43a72f7ec73f629514', 'request_id': 'req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:54.536844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2af625f97834e43a72f7ec73f629514', 'request_id': 'req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:54.560660 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] The token cache is being invalidate because project a2af625f97834e43a72f7ec73f629514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:54.561121 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:54.562159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:54.567061 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1362/2756] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:54 2022] DELETE /identity/v3/projects/a2af625f97834e43a72f7ec73f629514 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:54.568763 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f397f-8521-4bec-993c-c728ff6ba225 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:54.569293 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f397f-8521-4bec-993c-c728ff6ba225 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:54.569398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f397f-8521-4bec-993c-c728ff6ba225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:54.609085 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-d35f397f-8521-4bec-993c-c728ff6ba225 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:54.610287 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d35f397f-8521-4bec-993c-c728ff6ba225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:54.616434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:54.618500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:54.701629 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:54.731620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1395/2757] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:54.733145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3236c1b-f7af-40e6-b047-3d9f081c18dc None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:54.735569 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3236c1b-f7af-40e6-b047-3d9f081c18dc None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:54.736040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3236c1b-f7af-40e6-b047-3d9f081c18dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:54.772432 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb216ad-f6d9-48ba-9b7c-3d90417b0720 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:54.779556 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-d3236c1b-f7af-40e6-b047-3d9f081c18dc None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:54.780811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d3236c1b-f7af-40e6-b047-3d9f081c18dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:54.785006 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5d338-d8bb-4359-9993-4a72251192a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:54.785880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5d338-d8bb-4359-9993-4a72251192a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:54.786334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5d338-d8bb-4359-9993-4a72251192a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:54.786612 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5d338-d8bb-4359-9993-4a72251192a8 admin admin] PATH_INFO: `/v3/users/807ee9adf7d44276829cf53e531151af` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:54.789299 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5d338-d8bb-4359-9993-4a72251192a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=807ee9adf7d44276829cf53e531151af)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:54.791928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5d338-d8bb-4359-9993-4a72251192a8 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:54.811230 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1396/2758] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:54.815245 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3236c1b-f7af-40e6-b047-3d9f081c18dc None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:54.827961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3236c1b-f7af-40e6-b047-3d9f081c18dc None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:54.840313 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e908334-61db-434b-bd18-2dfa69ad6ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:54.841603 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e908334-61db-434b-bd18-2dfa69ad6ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:54.844065 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e908334-61db-434b-bd18-2dfa69ad6ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:54.844539 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e908334-61db-434b-bd18-2dfa69ad6ae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:54.845418 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e908334-61db-434b-bd18-2dfa69ad6ae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:54.845842 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e908334-61db-434b-bd18-2dfa69ad6ae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:54.847752 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e908334-61db-434b-bd18-2dfa69ad6ae1 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:54.862505 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-23d5d338-d8bb-4359-9993-4a72251192a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '807ee9adf7d44276829cf53e531151af'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:54.873220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-23d5d338-d8bb-4359-9993-4a72251192a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '807ee9adf7d44276829cf53e531151af'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:54.873220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-23d5d338-d8bb-4359-9993-4a72251192a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '807ee9adf7d44276829cf53e531151af'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:54.873220 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-23d5d338-d8bb-4359-9993-4a72251192a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '807ee9adf7d44276829cf53e531151af'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:54.875114 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1397/2759] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 14:59:54.876590 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e908334-61db-434b-bd18-2dfa69ad6ae1 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:54.879167 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-1e908334-61db-434b-bd18-2dfa69ad6ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:54.887304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-23d5d338-d8bb-4359-9993-4a72251192a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '807ee9adf7d44276829cf53e531151af'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:54.910144 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e908334-61db-434b-bd18-2dfa69ad6ae1 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:54.912157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:54.915654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a1950-8c32-4f15-98f7-ed5bcc0267be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:54.916320 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a1950-8c32-4f15-98f7-ed5bcc0267be admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:54.916596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a1950-8c32-4f15-98f7-ed5bcc0267be admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:54.916865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a1950-8c32-4f15-98f7-ed5bcc0267be admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:54.918152 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a1950-8c32-4f15-98f7-ed5bcc0267be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:54.921416 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a1950-8c32-4f15-98f7-ed5bcc0267be admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:54.933411 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1363/2760] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:54 2022] DELETE /identity/v3/users/807ee9adf7d44276829cf53e531151af => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:54.940393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bba3ef-a99d-411e-9549-5b2e9b275d53 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:54.941507 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bba3ef-a99d-411e-9549-5b2e9b275d53 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:54.942574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bba3ef-a99d-411e-9549-5b2e9b275d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:54.983494 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-07bba3ef-a99d-411e-9549-5b2e9b275d53 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:54.986054 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-07bba3ef-a99d-411e-9549-5b2e9b275d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:54.989481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1398/2761] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:54 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:54.994733 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a1950-8c32-4f15-98f7-ed5bcc0267be admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:55.117375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a1950-8c32-4f15-98f7-ed5bcc0267be admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:55.123310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386aecd3-6b0b-415c-ba09-64d35b0fab8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:55.124406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386aecd3-6b0b-415c-ba09-64d35b0fab8d admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:55.124787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386aecd3-6b0b-415c-ba09-64d35b0fab8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:55.126392 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386aecd3-6b0b-415c-ba09-64d35b0fab8d admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:55.127512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386aecd3-6b0b-415c-ba09-64d35b0fab8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:55.129582 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386aecd3-6b0b-415c-ba09-64d35b0fab8d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:55.138494 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1364/2762] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:55.149624 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bba3ef-a99d-411e-9549-5b2e9b275d53 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:55.151766 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-07bba3ef-a99d-411e-9549-5b2e9b275d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:55.176969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1399/2763] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:54 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:59:55.180432 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386aecd3-6b0b-415c-ba09-64d35b0fab8d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:55.199309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386aecd3-6b0b-415c-ba09-64d35b0fab8d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:55.217497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2a7c2-8586-49e4-9cda-0a7c140add5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:55.218329 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2a7c2-8586-49e4-9cda-0a7c140add5a admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:55.219155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2a7c2-8586-49e4-9cda-0a7c140add5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:55.219155 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2a7c2-8586-49e4-9cda-0a7c140add5a admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:55.219986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2a7c2-8586-49e4-9cda-0a7c140add5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:55.222198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2a7c2-8586-49e4-9cda-0a7c140add5a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:55.245279 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1400/2764] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:55.246660 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2a7c2-8586-49e4-9cda-0a7c140add5a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:55.266281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2a7c2-8586-49e4-9cda-0a7c140add5a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:55.276508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2136cf-211e-450c-b674-54069d3a5a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:55.277314 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2136cf-211e-450c-b674-54069d3a5a80 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:55.277692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2136cf-211e-450c-b674-54069d3a5a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:55.278063 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2136cf-211e-450c-b674-54069d3a5a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:55.278816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2136cf-211e-450c-b674-54069d3a5a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:55.280598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2136cf-211e-450c-b674-54069d3a5a80 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:55.303127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1401/2765] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:55.305398 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2136cf-211e-450c-b674-54069d3a5a80 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:55.324551 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2136cf-211e-450c-b674-54069d3a5a80 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:55.330222 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bba3ef-a99d-411e-9549-5b2e9b275d53 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:55.332356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8b6e2-bded-4af5-991c-52d8b683eade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:55.333139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b6e2-bded-4af5-991c-52d8b683eade admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:55.333517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b6e2-bded-4af5-991c-52d8b683eade admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:55.333889 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b6e2-bded-4af5-991c-52d8b683eade admin admin] PATH_INFO: `/v3/projects/c94caf834a354247bed0d9ec95b87407/users/1212dc2d9d0442369b8fa878a2645904/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:55.338498 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8e73d-e1d4-4892-9168-93248f4b9c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:55.339508 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8e73d-e1d4-4892-9168-93248f4b9c9d service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:55.339855 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8e73d-e1d4-4892-9168-93248f4b9c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:55.340138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8e73d-e1d4-4892-9168-93248f4b9c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:55.352019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8e73d-e1d4-4892-9168-93248f4b9c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:55.353496 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8e73d-e1d4-4892-9168-93248f4b9c9d service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:55.366380 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1365/2766] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:59:55.374437 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8b6e2-bded-4af5-991c-52d8b683eade admin admin] RBAC: Authorizing `identity:create_grant(project_id=c94caf834a354247bed0d9ec95b87407, user_id=1212dc2d9d0442369b8fa878a2645904, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:55.377664 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8b6e2-bded-4af5-991c-52d8b683eade admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:55.397028 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1402/2767] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:55 2022] PUT /identity/v3/projects/c94caf834a354247bed0d9ec95b87407/users/1212dc2d9d0442369b8fa878a2645904/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:55.407995 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c21a4d-697b-4096-88b3-939114b991e3 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:55.408290 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c21a4d-697b-4096-88b3-939114b991e3 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:55.408578 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c21a4d-697b-4096-88b3-939114b991e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:55.449497 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-d8c21a4d-697b-4096-88b3-939114b991e3 None None] MFA Rules not processed for user `1212dc2d9d0442369b8fa878a2645904`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:55.452352 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d8c21a4d-697b-4096-88b3-939114b991e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:55.606350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1366/2768] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:55.610655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8b6e2-bded-4af5-991c-52d8b683eade admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:55.760056 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:55.780060 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8b6e2-bded-4af5-991c-52d8b683eade admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:55.796440 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6f8a4-d656-48d0-982e-d19aac6b7ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:55.797091 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6f8a4-d656-48d0-982e-d19aac6b7ffb admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:55.797374 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6f8a4-d656-48d0-982e-d19aac6b7ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:55.797647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6f8a4-d656-48d0-982e-d19aac6b7ffb admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:55.798349 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6f8a4-d656-48d0-982e-d19aac6b7ffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:55.805167 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6f8a4-d656-48d0-982e-d19aac6b7ffb admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:55.863802 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1403/2769] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:55 2022] POST /identity/v3/projects => generated 385 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:55.870811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c21a4d-697b-4096-88b3-939114b991e3 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:55.887638 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c21a4d-697b-4096-88b3-939114b991e3 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:55.955794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:55.980598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6f8a4-d656-48d0-982e-d19aac6b7ffb admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:55.983081 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f8d6f8a4-d656-48d0-982e-d19aac6b7ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:56.044337 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1febb1-09a2-42ec-b272-0a2cca9683b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.045021 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1febb1-09a2-42ec-b272-0a2cca9683b2 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.045315 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1febb1-09a2-42ec-b272-0a2cca9683b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.045601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1febb1-09a2-42ec-b272-0a2cca9683b2 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.048559 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1febb1-09a2-42ec-b272-0a2cca9683b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.052648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1febb1-09a2-42ec-b272-0a2cca9683b2 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.092042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6f8a4-d656-48d0-982e-d19aac6b7ffb admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.098347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a21eb-6414-4e18-9dee-bf65e81ef971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.098997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a21eb-6414-4e18-9dee-bf65e81ef971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.099346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a21eb-6414-4e18-9dee-bf65e81ef971 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.099710 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a21eb-6414-4e18-9dee-bf65e81ef971 admin admin] PATH_INFO: `/v3/projects/56222de93bf841ed82f6683c1b1f1cf9` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.101623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a21eb-6414-4e18-9dee-bf65e81ef971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56222de93bf841ed82f6683c1b1f1cf9)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.103953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a21eb-6414-4e18-9dee-bf65e81ef971 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.109040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1367/2770] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:55 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:56.110280 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1febb1-09a2-42ec-b272-0a2cca9683b2 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.137972 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1febb1-09a2-42ec-b272-0a2cca9683b2 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.144641 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67ae9c-106a-4f43-a0b0-d573b1779338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.150665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67ae9c-106a-4f43-a0b0-d573b1779338 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.151379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67ae9c-106a-4f43-a0b0-d573b1779338 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.151961 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67ae9c-106a-4f43-a0b0-d573b1779338 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.154370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67ae9c-106a-4f43-a0b0-d573b1779338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.158223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67ae9c-106a-4f43-a0b0-d573b1779338 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.176404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1368/2771] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:56.183362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67ae9c-106a-4f43-a0b0-d573b1779338 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.184107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-717a21eb-6414-4e18-9dee-bf65e81ef971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56222de93bf841ed82f6683c1b1f1cf9', 'request_id': 'req-717a21eb-6414-4e18-9dee-bf65e81ef971'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:56.196294 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67ae9c-106a-4f43-a0b0-d573b1779338 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.197395 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-717a21eb-6414-4e18-9dee-bf65e81ef971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56222de93bf841ed82f6683c1b1f1cf9', 'request_id': 'req-717a21eb-6414-4e18-9dee-bf65e81ef971'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:56.214239 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-717a21eb-6414-4e18-9dee-bf65e81ef971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56222de93bf841ed82f6683c1b1f1cf9', 'request_id': 'req-717a21eb-6414-4e18-9dee-bf65e81ef971'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:56.239060 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-717a21eb-6414-4e18-9dee-bf65e81ef971 admin admin] The token cache is being invalidate because project 56222de93bf841ed82f6683c1b1f1cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:56.240224 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-717a21eb-6414-4e18-9dee-bf65e81ef971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:56.241148 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-717a21eb-6414-4e18-9dee-bf65e81ef971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:56.244563 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1404/2772] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:55 2022] DELETE /identity/v3/projects/56222de93bf841ed82f6683c1b1f1cf9 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:56.251388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d452-083c-4cb5-acc1-2ede273fbcb7 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.251953 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d452-083c-4cb5-acc1-2ede273fbcb7 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.252375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d452-083c-4cb5-acc1-2ede273fbcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.284615 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-83e4d452-083c-4cb5-acc1-2ede273fbcb7 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:56.287666 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6939a5-d9ad-4f2f-9d73-29644e9ec304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.289189 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6939a5-d9ad-4f2f-9d73-29644e9ec304 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.289455 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-83e4d452-083c-4cb5-acc1-2ede273fbcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:56.289664 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6939a5-d9ad-4f2f-9d73-29644e9ec304 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.290038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6939a5-d9ad-4f2f-9d73-29644e9ec304 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.290786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6939a5-d9ad-4f2f-9d73-29644e9ec304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.292599 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6939a5-d9ad-4f2f-9d73-29644e9ec304 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.309501 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1369/2773] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:56.310472 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6939a5-d9ad-4f2f-9d73-29644e9ec304 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.311136 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1405/2774] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:56.313231 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3d6939a5-d9ad-4f2f-9d73-29644e9ec304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:56.319959 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268fe89-02b2-4b73-98fb-33297773bd80 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.320257 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268fe89-02b2-4b73-98fb-33297773bd80 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.325865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268fe89-02b2-4b73-98fb-33297773bd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.339034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6939a5-d9ad-4f2f-9d73-29644e9ec304 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.344866 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0226e-676c-4d44-b92b-0fc4330a3623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.345647 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0226e-676c-4d44-b92b-0fc4330a3623 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.345932 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0226e-676c-4d44-b92b-0fc4330a3623 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.346208 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0226e-676c-4d44-b92b-0fc4330a3623 admin admin] PATH_INFO: `/v3/projects/8d0ff5ed2edf404f871aa66726b333b5/users/31a83c3bbce743ffb1e8eb2d698368ff/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.357427 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4268fe89-02b2-4b73-98fb-33297773bd80 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:56.358960 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4268fe89-02b2-4b73-98fb-33297773bd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:56.375702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0226e-676c-4d44-b92b-0fc4330a3623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0ff5ed2edf404f871aa66726b333b5, user_id=31a83c3bbce743ffb1e8eb2d698368ff, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.375702 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0226e-676c-4d44-b92b-0fc4330a3623 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.382083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1406/2775] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:56.388717 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268fe89-02b2-4b73-98fb-33297773bd80 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.399550 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268fe89-02b2-4b73-98fb-33297773bd80 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.400218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1370/2776] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:56 2022] PUT /identity/v3/projects/8d0ff5ed2edf404f871aa66726b333b5/users/31a83c3bbce743ffb1e8eb2d698368ff/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:56.402458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd9d00-041e-431e-9363-2a79d4fa295b None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.403010 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd9d00-041e-431e-9363-2a79d4fa295b None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.403143 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd9d00-041e-431e-9363-2a79d4fa295b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.437679 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-00fd9d00-041e-431e-9363-2a79d4fa295b None None] MFA Rules not processed for user `31a83c3bbce743ffb1e8eb2d698368ff`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:56.439014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-00fd9d00-041e-431e-9363-2a79d4fa295b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:56.487975 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:56.515032 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69607b-6dce-4123-aafd-910a13c7f4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.515759 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69607b-6dce-4123-aafd-910a13c7f4fe admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.516043 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69607b-6dce-4123-aafd-910a13c7f4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.517350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69607b-6dce-4123-aafd-910a13c7f4fe admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.517350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69607b-6dce-4123-aafd-910a13c7f4fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 14:59:56.517350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69607b-6dce-4123-aafd-910a13c7f4fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.519453 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69607b-6dce-4123-aafd-910a13c7f4fe admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.543502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1407/2777] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 14:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 14:59:56.545479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69607b-6dce-4123-aafd-910a13c7f4fe admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.555531 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1371/2778] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:56.559948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd9d00-041e-431e-9363-2a79d4fa295b None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.560767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69607b-6dce-4123-aafd-910a13c7f4fe admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.567008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f33331-4312-40df-b6f1-eb228f9f9253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.567697 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f33331-4312-40df-b6f1-eb228f9f9253 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.567996 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f33331-4312-40df-b6f1-eb228f9f9253 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.568290 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f33331-4312-40df-b6f1-eb228f9f9253 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.568896 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f33331-4312-40df-b6f1-eb228f9f9253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.571598 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f33331-4312-40df-b6f1-eb228f9f9253 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.582162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd9d00-041e-431e-9363-2a79d4fa295b None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.591665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0791356-67b2-4eda-ace3-18580e0767f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.593228 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0791356-67b2-4eda-ace3-18580e0767f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.594106 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0791356-67b2-4eda-ace3-18580e0767f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.594503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0791356-67b2-4eda-ace3-18580e0767f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.596063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0791356-67b2-4eda-ace3-18580e0767f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.598689 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0791356-67b2-4eda-ace3-18580e0767f4 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.613223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:56.618142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1408/2779] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:56 2022] POST /identity/v3/projects => generated 369 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:56.622786 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f33331-4312-40df-b6f1-eb228f9f9253 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.638795 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1372/2780] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 14:59:56 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 14:59:56.643334 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0791356-67b2-4eda-ace3-18580e0767f4 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.733188 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0791356-67b2-4eda-ace3-18580e0767f4 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.736465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f33331-4312-40df-b6f1-eb228f9f9253 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.743935 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90df8-9181-4438-9212-76513e18a6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.744241 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90df8-9181-4438-9212-76513e18a6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.745125 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90df8-9181-4438-9212-76513e18a6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.745409 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90df8-9181-4438-9212-76513e18a6a3 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.746066 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e90df8-9181-4438-9212-76513e18a6a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.748043 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e90df8-9181-4438-9212-76513e18a6a3 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.752237 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ed6f8-66a6-49de-9ef4-4812b5e80795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.752944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ed6f8-66a6-49de-9ef4-4812b5e80795 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.753378 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ed6f8-66a6-49de-9ef4-4812b5e80795 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.753656 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ed6f8-66a6-49de-9ef4-4812b5e80795 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.754278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ed6f8-66a6-49de-9ef4-4812b5e80795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.757459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ed6f8-66a6-49de-9ef4-4812b5e80795 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.810602 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1409/2781] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:56 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:56.811440 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1373/2782] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 14:59:56 2022] POST /identity/v3/users => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 14:59:56.812012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ed6f8-66a6-49de-9ef4-4812b5e80795 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.818986 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90df8-9181-4438-9212-76513e18a6a3 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.833986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ed6f8-66a6-49de-9ef4-4812b5e80795 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.855413 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e91fda-b40f-42fb-bbaf-2c87c47551fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.858354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e91fda-b40f-42fb-bbaf-2c87c47551fe admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.858640 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e91fda-b40f-42fb-bbaf-2c87c47551fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.858926 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e91fda-b40f-42fb-bbaf-2c87c47551fe admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.859600 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e91fda-b40f-42fb-bbaf-2c87c47551fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.864923 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90df8-9181-4438-9212-76513e18a6a3 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.865966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e91fda-b40f-42fb-bbaf-2c87c47551fe admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.881370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95749f-2996-42a5-8605-ad3bb29b9204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.882323 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95749f-2996-42a5-8605-ad3bb29b9204 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.882615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95749f-2996-42a5-8605-ad3bb29b9204 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.883985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95749f-2996-42a5-8605-ad3bb29b9204 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.887544 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95749f-2996-42a5-8605-ad3bb29b9204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.889223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95749f-2996-42a5-8605-ad3bb29b9204 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.897182 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1410/2783] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 14:59:56.900441 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e91fda-b40f-42fb-bbaf-2c87c47551fe admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.925404 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1374/2784] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:56.930301 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95749f-2996-42a5-8605-ad3bb29b9204 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.933337 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e91fda-b40f-42fb-bbaf-2c87c47551fe admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.938774 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ef411-1679-498f-94fd-091a4037a23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.939406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ef411-1679-498f-94fd-091a4037a23b admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.939734 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ef411-1679-498f-94fd-091a4037a23b admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.941623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ef411-1679-498f-94fd-091a4037a23b admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.942406 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ef411-1679-498f-94fd-091a4037a23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.946466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ef411-1679-498f-94fd-091a4037a23b admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:56.953783 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95749f-2996-42a5-8605-ad3bb29b9204 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:56.959469 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9803dfe-22ba-4b4f-b4bb-336eeffdeceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:56.960247 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9803dfe-22ba-4b4f-b4bb-336eeffdeceb admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:56.961037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9803dfe-22ba-4b4f-b4bb-336eeffdeceb admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:56.961327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9803dfe-22ba-4b4f-b4bb-336eeffdeceb admin admin] PATH_INFO: `/v3/projects/c3f84feae23345efbeea2f244472d5fa/users/6d4ea3cf697444ccaaddf95ab6bb6b00/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:56.978247 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1411/2785] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:56.979227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ef411-1679-498f-94fd-091a4037a23b admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:56.985449 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9803dfe-22ba-4b4f-b4bb-336eeffdeceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f84feae23345efbeea2f244472d5fa, user_id=6d4ea3cf697444ccaaddf95ab6bb6b00, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:56.989728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9803dfe-22ba-4b4f-b4bb-336eeffdeceb admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:57.003535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ef411-1679-498f-94fd-091a4037a23b admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:57.010195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1375/2786] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:56 2022] PUT /identity/v3/projects/c3f84feae23345efbeea2f244472d5fa/users/6d4ea3cf697444ccaaddf95ab6bb6b00/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:57.014471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9803dfe-22ba-4b4f-b4bb-336eeffdeceb admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:57.123906 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d451ba-70ce-4d84-9934-7ecd19d0417a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:57.124596 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d451ba-70ce-4d84-9934-7ecd19d0417a admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:57.124882 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d451ba-70ce-4d84-9934-7ecd19d0417a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:57.125156 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d451ba-70ce-4d84-9934-7ecd19d0417a admin admin] PATH_INFO: `/v3/projects/7da1108748f34ab7baf4b92f33ce51ca/users/85b93477d3254db38f2fcc0bf2091190/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:57.127933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9803dfe-22ba-4b4f-b4bb-336eeffdeceb admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:57.136895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04a2e7-957e-453c-a5c9-e1535efc89c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:57.137662 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04a2e7-957e-453c-a5c9-e1535efc89c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:57.138057 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04a2e7-957e-453c-a5c9-e1535efc89c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:57.139040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04a2e7-957e-453c-a5c9-e1535efc89c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:57.140826 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04a2e7-957e-453c-a5c9-e1535efc89c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:57.143737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04a2e7-957e-453c-a5c9-e1535efc89c4 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:57.162535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1376/2787] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 14:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 14:59:57.162655 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d451ba-70ce-4d84-9934-7ecd19d0417a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7da1108748f34ab7baf4b92f33ce51ca, user_id=85b93477d3254db38f2fcc0bf2091190, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:57.163665 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04a2e7-957e-453c-a5c9-e1535efc89c4 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:57.168658 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d451ba-70ce-4d84-9934-7ecd19d0417a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:57.174899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04a2e7-957e-453c-a5c9-e1535efc89c4 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:57.182228 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6cbb8-2a7b-41de-bc6a-e9d7f2f8007a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:57.183157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6cbb8-2a7b-41de-bc6a-e9d7f2f8007a admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:57.183551 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6cbb8-2a7b-41de-bc6a-e9d7f2f8007a admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:57.184622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6cbb8-2a7b-41de-bc6a-e9d7f2f8007a admin admin] PATH_INFO: `/v3/projects/c3f84feae23345efbeea2f244472d5fa/users/6d4ea3cf697444ccaaddf95ab6bb6b00/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:57.187155 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6cbb8-2a7b-41de-bc6a-e9d7f2f8007a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f84feae23345efbeea2f244472d5fa, user_id=6d4ea3cf697444ccaaddf95ab6bb6b00, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:57.190197 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6cbb8-2a7b-41de-bc6a-e9d7f2f8007a admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:57.204223 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1412/2788] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:56 2022] PUT /identity/v3/projects/7da1108748f34ab7baf4b92f33ce51ca/users/85b93477d3254db38f2fcc0bf2091190/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:57.205816 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23135f5-60a1-4193-b95a-6795e1db1cfb None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:57.206105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23135f5-60a1-4193-b95a-6795e1db1cfb None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:57.206379 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23135f5-60a1-4193-b95a-6795e1db1cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:57.229677 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1377/2789] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 14:59:57 2022] PUT /identity/v3/projects/c3f84feae23345efbeea2f244472d5fa/users/6d4ea3cf697444ccaaddf95ab6bb6b00/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:57.233196 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b09d4-397e-4ef4-99d1-d06392df52ff None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:57.233485 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b09d4-397e-4ef4-99d1-d06392df52ff None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:57.233758 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b09d4-397e-4ef4-99d1-d06392df52ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:57.266140 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-b23135f5-60a1-4193-b95a-6795e1db1cfb None None] MFA Rules not processed for user `85b93477d3254db38f2fcc0bf2091190`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:57.267426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b23135f5-60a1-4193-b95a-6795e1db1cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:57.271293 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-0e4b09d4-397e-4ef4-99d1-d06392df52ff None None] MFA Rules not processed for user `6d4ea3cf697444ccaaddf95ab6bb6b00`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:57.273221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-0e4b09d4-397e-4ef4-99d1-d06392df52ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:57.278692 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:57.311981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:57.360964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1413/2790] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:57.370836 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23135f5-60a1-4193-b95a-6795e1db1cfb None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:57.373096 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-b23135f5-60a1-4193-b95a-6795e1db1cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:57.384426 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1378/2791] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 14:59:57.397019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b09d4-397e-4ef4-99d1-d06392df52ff None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:57.399969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-0e4b09d4-397e-4ef4-99d1-d06392df52ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:57.517384 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23135f5-60a1-4193-b95a-6795e1db1cfb None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:57.526741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b09d4-397e-4ef4-99d1-d06392df52ff None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:57.536891 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35b979-f656-499e-9649-79fd99b8b9a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:57.537289 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd70b2-85f0-4ca8-8eaf-e85a86fce62f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:57.538703 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35b979-f656-499e-9649-79fd99b8b9a8 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:57.538841 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd70b2-85f0-4ca8-8eaf-e85a86fce62f service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:57.538951 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35b979-f656-499e-9649-79fd99b8b9a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:57.539059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd70b2-85f0-4ca8-8eaf-e85a86fce62f service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:57.539162 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35b979-f656-499e-9649-79fd99b8b9a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:57.539262 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd70b2-85f0-4ca8-8eaf-e85a86fce62f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:57.541458 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7d35b979-f656-499e-9649-79fd99b8b9a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:57.553067 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd70b2-85f0-4ca8-8eaf-e85a86fce62f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:57.555196 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd70b2-85f0-4ca8-8eaf-e85a86fce62f service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:57.565352 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1414/2792] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:59:57.615024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd70b2-85f0-4ca8-8eaf-e85a86fce62f service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:57.628767 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35b979-f656-499e-9649-79fd99b8b9a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:57.630346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35b979-f656-499e-9649-79fd99b8b9a8 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:57.644075 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1379/2793] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:57.714268 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd70b2-85f0-4ca8-8eaf-e85a86fce62f service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:57.725544 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dabfe4-deca-42f1-a427-0f64eae23b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:57.726199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dabfe4-deca-42f1-a427-0f64eae23b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:57.726667 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dabfe4-deca-42f1-a427-0f64eae23b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:57.727012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dabfe4-deca-42f1-a427-0f64eae23b66 admin admin] PATH_INFO: `/v3/users/85b93477d3254db38f2fcc0bf2091190` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:57.730966 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dabfe4-deca-42f1-a427-0f64eae23b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b93477d3254db38f2fcc0bf2091190)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:57.734914 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dabfe4-deca-42f1-a427-0f64eae23b66 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:57.812259 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:57.840811 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-38dabfe4-deca-42f1-a427-0f64eae23b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b93477d3254db38f2fcc0bf2091190'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:57.857972 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-38dabfe4-deca-42f1-a427-0f64eae23b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b93477d3254db38f2fcc0bf2091190'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:57.858861 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-38dabfe4-deca-42f1-a427-0f64eae23b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b93477d3254db38f2fcc0bf2091190'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:57.906169 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-38dabfe4-deca-42f1-a427-0f64eae23b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b93477d3254db38f2fcc0bf2091190'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:57.917535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-38dabfe4-deca-42f1-a427-0f64eae23b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b93477d3254db38f2fcc0bf2091190'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:57.926747 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1415/2794] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 14:59:57 2022] DELETE /identity/v3/users/85b93477d3254db38f2fcc0bf2091190 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 14:59:57.934598 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776085d2-790b-4648-9d32-d37dc00bd1f9 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:57.935170 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776085d2-790b-4648-9d32-d37dc00bd1f9 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:57.935292 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776085d2-790b-4648-9d32-d37dc00bd1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:57.967582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-776085d2-790b-4648-9d32-d37dc00bd1f9 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:57.969193 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-776085d2-790b-4648-9d32-d37dc00bd1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:58.037518 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9fae5-5b92-4e45-b973-e482dc12945a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:58.038242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9fae5-5b92-4e45-b973-e482dc12945a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:58.039835 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9fae5-5b92-4e45-b973-e482dc12945a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:58.069993 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:58.082864 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-afe9fae5-5b92-4e45-b973-e482dc12945a None None] MFA Rules not processed for user `e105e9d96c7d4cc296745ee791efaa2e`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:58.084684 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-afe9fae5-5b92-4e45-b973-e482dc12945a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:58.090046 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1380/2795] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:58.100717 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776085d2-790b-4648-9d32-d37dc00bd1f9 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:58.102369 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-776085d2-790b-4648-9d32-d37dc00bd1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:58.190258 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776085d2-790b-4648-9d32-d37dc00bd1f9 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:58.196991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3938001-d2bb-4d53-bab0-04fc990a864d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:58.198416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3938001-d2bb-4d53-bab0-04fc990a864d service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:58.199008 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3938001-d2bb-4d53-bab0-04fc990a864d service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:58.199743 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3938001-d2bb-4d53-bab0-04fc990a864d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:58.211654 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1416/2796] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 14:59:58.219732 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9fae5-5b92-4e45-b973-e482dc12945a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:58.221311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-afe9fae5-5b92-4e45-b973-e482dc12945a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:58.223893 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3938001-d2bb-4d53-bab0-04fc990a864d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:58.225925 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3938001-d2bb-4d53-bab0-04fc990a864d service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:58.238137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1381/2797] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:59:58.375315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9fae5-5b92-4e45-b973-e482dc12945a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:58.388008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ddd96-6d48-43fc-a577-bef31dff7ac8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:58.388883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ddd96-6d48-43fc-a577-bef31dff7ac8 service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:58.389258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ddd96-6d48-43fc-a577-bef31dff7ac8 service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:58.390069 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ddd96-6d48-43fc-a577-bef31dff7ac8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:58.392552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ac1ddd96-6d48-43fc-a577-bef31dff7ac8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:58.402346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 14:59:58.479873 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ddd96-6d48-43fc-a577-bef31dff7ac8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:58.481500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ddd96-6d48-43fc-a577-bef31dff7ac8 service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:58.494466 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1417/2798] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 14:59:58.569800 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3938001-d2bb-4d53-bab0-04fc990a864d service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:58.571915 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-a3938001-d2bb-4d53-bab0-04fc990a864d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:58.591324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3938001-d2bb-4d53-bab0-04fc990a864d service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:58.602628 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:58.603295 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:58.603724 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:58.603900 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1 admin admin] PATH_INFO: `/v3/projects/7da1108748f34ab7baf4b92f33ce51ca` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:58.609927 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7da1108748f34ab7baf4b92f33ce51ca)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:58.612730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:58.680940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7da1108748f34ab7baf4b92f33ce51ca', 'request_id': 'req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:58.686822 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7da1108748f34ab7baf4b92f33ce51ca', 'request_id': 'req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:58.696108 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7da1108748f34ab7baf4b92f33ce51ca', 'request_id': 'req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:58.708078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1 admin admin] The token cache is being invalidate because project 7da1108748f34ab7baf4b92f33ce51ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 14:59:58.708378 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:58.709040 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-ff9a0622-c875-4f9c-82d5-0dc87829b8e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 14:59:58.711165 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1382/2799] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 14:59:58 2022] DELETE /identity/v3/projects/7da1108748f34ab7baf4b92f33ce51ca => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 14:59:59.423098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ddd96-6d48-43fc-a577-bef31dff7ac8 service nova] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 14:59:59.424661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ac1ddd96-6d48-43fc-a577-bef31dff7ac8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:59.491883 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ddd96-6d48-43fc-a577-bef31dff7ac8 service nova] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 14:59:59.556340 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1dd9b-60ca-467b-ac11-23a046ec4318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 14:59:59.556977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1dd9b-60ca-467b-ac11-23a046ec4318 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:59.557261 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1dd9b-60ca-467b-ac11-23a046ec4318 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:59.557537 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1dd9b-60ca-467b-ac11-23a046ec4318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:59.559420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-22c1dd9b-60ca-467b-ac11-23a046ec4318 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:59.642494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1dd9b-60ca-467b-ac11-23a046ec4318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 14:59:59.644198 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1dd9b-60ca-467b-ac11-23a046ec4318 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 14:59:59.658983 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1418/2800] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 14:59:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 14:59:59.767777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142e1d5-bbdd-40cc-b2d1-b7c4969d570f None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 14:59:59.768875 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142e1d5-bbdd-40cc-b2d1-b7c4969d570f None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 14:59:59.769235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142e1d5-bbdd-40cc-b2d1-b7c4969d570f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 14:59:59.792495 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-d142e1d5-bbdd-40cc-b2d1-b7c4969d570f None None] MFA Rules not processed for user `1e3a4f725f3d41edbb7c162a8760bf8b`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 14:59:59.794325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d142e1d5-bbdd-40cc-b2d1-b7c4969d570f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 14:59:59.879038 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1383/2801] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:00.396575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1dd9b-60ca-467b-ac11-23a046ec4318 service cinder] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:00.398609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-22c1dd9b-60ca-467b-ac11-23a046ec4318 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:00.412377 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1dd9b-60ca-467b-ac11-23a046ec4318 service cinder] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:00.419872 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67403e16-4c13-4af9-81e6-09826216ef37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:00.420738 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67403e16-4c13-4af9-81e6-09826216ef37 service cinder] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:00.421020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67403e16-4c13-4af9-81e6-09826216ef37 service cinder] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:00.421451 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67403e16-4c13-4af9-81e6-09826216ef37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:00.423481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-67403e16-4c13-4af9-81e6-09826216ef37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:00.455931 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:00.518923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67403e16-4c13-4af9-81e6-09826216ef37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:00.520587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67403e16-4c13-4af9-81e6-09826216ef37 service cinder] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:00.531240 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1419/2802] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:00:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:00:04.538962 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142e1d5-bbdd-40cc-b2d1-b7c4969d570f None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:04.540460 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d142e1d5-bbdd-40cc-b2d1-b7c4969d570f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:04.572178 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:04.607142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142e1d5-bbdd-40cc-b2d1-b7c4969d570f None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:04.611757 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:04.612405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:04.612679 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:04.612945 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] PATH_INFO: `/v3/users/1212dc2d9d0442369b8fa878a2645904` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:04.614051 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1212dc2d9d0442369b8fa878a2645904)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:04.616037 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:04.666844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1212dc2d9d0442369b8fa878a2645904'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:04.685844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1212dc2d9d0442369b8fa878a2645904'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:04.687357 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1212dc2d9d0442369b8fa878a2645904'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:04.689143 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1212dc2d9d0442369b8fa878a2645904'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:04.706230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1212dc2d9d0442369b8fa878a2645904'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:04.720427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1384/2803] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 15:00:04 2022] DELETE /identity/v3/users/1212dc2d9d0442369b8fa878a2645904 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:00:04.725294 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e53367-9fec-413a-824d-66e8f7e32e8d None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:04.725782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e53367-9fec-413a-824d-66e8f7e32e8d None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:04.726105 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e53367-9fec-413a-824d-66e8f7e32e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:04.745146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-30e53367-9fec-413a-824d-66e8f7e32e8d None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:04.746497 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-30e53367-9fec-413a-824d-66e8f7e32e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:04.812051 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1420/2804] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:04.822139 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:04.823591 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:04.889553 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c66cdd-7bad-4077-8245-5113b45e0fd7 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:04.893532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb4d41-513e-4a40-b080-08bfd9e5b93e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:04.894142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb4d41-513e-4a40-b080-08bfd9e5b93e service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:04.894419 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb4d41-513e-4a40-b080-08bfd9e5b93e service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:04.894692 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb4d41-513e-4a40-b080-08bfd9e5b93e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:04.904634 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb4d41-513e-4a40-b080-08bfd9e5b93e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:04.906103 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb4d41-513e-4a40-b080-08bfd9e5b93e service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:04.916471 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1385/2805] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:00:05.071981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e53367-9fec-413a-824d-66e8f7e32e8d None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:05.082361 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e53367-9fec-413a-824d-66e8f7e32e8d None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:05.086787 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650257f-d40b-4235-ab4f-48615be1e34e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:05.087508 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650257f-d40b-4235-ab4f-48615be1e34e service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:05.087886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650257f-d40b-4235-ab4f-48615be1e34e service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:05.088227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650257f-d40b-4235-ab4f-48615be1e34e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:05.090013 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-2650257f-d40b-4235-ab4f-48615be1e34e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:05.152887 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650257f-d40b-4235-ab4f-48615be1e34e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:05.154422 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650257f-d40b-4235-ab4f-48615be1e34e service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:05.166587 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1421/2806] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:00:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:00:05.201045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb4d41-513e-4a40-b080-08bfd9e5b93e service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:05.202511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-dbbb4d41-513e-4a40-b080-08bfd9e5b93e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:05.230709 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb4d41-513e-4a40-b080-08bfd9e5b93e service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:05.236089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:05.236732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:05.237068 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:05.237304 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] PATH_INFO: `/v3/projects/c94caf834a354247bed0d9ec95b87407` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:05.238533 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c94caf834a354247bed0d9ec95b87407)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:05.240648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:05.290556 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:05.293646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c94caf834a354247bed0d9ec95b87407', 'request_id': 'req-50549db3-6e73-4b57-beaf-abeec409e178'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.300221 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c94caf834a354247bed0d9ec95b87407', 'request_id': 'req-50549db3-6e73-4b57-beaf-abeec409e178'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.308574 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c94caf834a354247bed0d9ec95b87407', 'request_id': 'req-50549db3-6e73-4b57-beaf-abeec409e178'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.320473 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] The token cache is being invalidate because project c94caf834a354247bed0d9ec95b87407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 15:00:05.320688 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.321327 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.322794 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1386/2807] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 15:00:05 2022] DELETE /identity/v3/projects/c94caf834a354247bed0d9ec95b87407 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:00:05.326924 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650257f-d40b-4235-ab4f-48615be1e34e service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:05.343763 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650257f-d40b-4235-ab4f-48615be1e34e service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:05.394121 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:05.408177 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:05.409047 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:05.409446 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:05.409820 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] PATH_INFO: `/v3/users/31a83c3bbce743ffb1e8eb2d698368ff` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:05.411051 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31a83c3bbce743ffb1e8eb2d698368ff)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:05.413057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:05.459645 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31a83c3bbce743ffb1e8eb2d698368ff'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.467952 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31a83c3bbce743ffb1e8eb2d698368ff'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.468648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31a83c3bbce743ffb1e8eb2d698368ff'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.480773 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31a83c3bbce743ffb1e8eb2d698368ff'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.490426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31a83c3bbce743ffb1e8eb2d698368ff'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.499254 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1422/2808] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 15:00:05 2022] DELETE /identity/v3/users/31a83c3bbce743ffb1e8eb2d698368ff => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:00:05.796034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:05.797545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:05.813014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50549db3-6e73-4b57-beaf-abeec409e178 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:05.874970 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:05.875513 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:05.875830 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:05.876110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] PATH_INFO: `/v3/projects/8d0ff5ed2edf404f871aa66726b333b5` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:05.877237 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d0ff5ed2edf404f871aa66726b333b5)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:05.879143 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:05.946387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d0ff5ed2edf404f871aa66726b333b5', 'request_id': 'req-223e85a7-ae5c-4cf4-a045-52edb55ba36a'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.952387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d0ff5ed2edf404f871aa66726b333b5', 'request_id': 'req-223e85a7-ae5c-4cf4-a045-52edb55ba36a'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.960548 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d0ff5ed2edf404f871aa66726b333b5', 'request_id': 'req-223e85a7-ae5c-4cf4-a045-52edb55ba36a'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.972451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] The token cache is being invalidate because project 8d0ff5ed2edf404f871aa66726b333b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 15:00:05.972742 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.973386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:05.974907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1387/2809] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 15:00:05 2022] DELETE /identity/v3/projects/8d0ff5ed2edf404f871aa66726b333b5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 15:00:05.979562 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:05.981368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:05.995522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c19e2-67ce-40e2-8fef-67db88ced5b5 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:06.063368 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606350e-b973-4255-a897-bcf9ec42ad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:06.064166 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606350e-b973-4255-a897-bcf9ec42ad41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:06.064571 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606350e-b973-4255-a897-bcf9ec42ad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:06.064853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606350e-b973-4255-a897-bcf9ec42ad41 admin admin] PATH_INFO: `/v3/users/6d4ea3cf697444ccaaddf95ab6bb6b00` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:06.066042 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606350e-b973-4255-a897-bcf9ec42ad41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d4ea3cf697444ccaaddf95ab6bb6b00)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:06.068175 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606350e-b973-4255-a897-bcf9ec42ad41 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:06.120584 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0606350e-b973-4255-a897-bcf9ec42ad41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d4ea3cf697444ccaaddf95ab6bb6b00'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:06.128999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0606350e-b973-4255-a897-bcf9ec42ad41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d4ea3cf697444ccaaddf95ab6bb6b00'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:06.129638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0606350e-b973-4255-a897-bcf9ec42ad41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d4ea3cf697444ccaaddf95ab6bb6b00'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:06.138494 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:06.144209 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0606350e-b973-4255-a897-bcf9ec42ad41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d4ea3cf697444ccaaddf95ab6bb6b00'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:06.152993 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-0606350e-b973-4255-a897-bcf9ec42ad41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d4ea3cf697444ccaaddf95ab6bb6b00'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:06.159318 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1423/2810] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 15:00:05 2022] DELETE /identity/v3/users/6d4ea3cf697444ccaaddf95ab6bb6b00 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 15:00:06.593869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:06.595427 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:06.608500 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e85a7-ae5c-4cf4-a045-52edb55ba36a admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:06.670085 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8966265-e638-4834-aa6c-17fe77540f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:06.670756 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8966265-e638-4834-aa6c-17fe77540f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:06.671035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8966265-e638-4834-aa6c-17fe77540f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:06.671306 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8966265-e638-4834-aa6c-17fe77540f2e admin admin] PATH_INFO: `/v3/projects/c3f84feae23345efbeea2f244472d5fa` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:06.672462 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8966265-e638-4834-aa6c-17fe77540f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f84feae23345efbeea2f244472d5fa)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:06.674351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8966265-e638-4834-aa6c-17fe77540f2e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:06.682982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:06.750499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f8966265-e638-4834-aa6c-17fe77540f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f84feae23345efbeea2f244472d5fa', 'request_id': 'req-f8966265-e638-4834-aa6c-17fe77540f2e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:06.758446 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f8966265-e638-4834-aa6c-17fe77540f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f84feae23345efbeea2f244472d5fa', 'request_id': 'req-f8966265-e638-4834-aa6c-17fe77540f2e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:06.767197 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f8966265-e638-4834-aa6c-17fe77540f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f84feae23345efbeea2f244472d5fa', 'request_id': 'req-f8966265-e638-4834-aa6c-17fe77540f2e'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:06.787736 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f8966265-e638-4834-aa6c-17fe77540f2e admin admin] The token cache is being invalidate because project c3f84feae23345efbeea2f244472d5fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 15:00:06.788563 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f8966265-e638-4834-aa6c-17fe77540f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:06.791028 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-f8966265-e638-4834-aa6c-17fe77540f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:06.792582 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1388/2811] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 15:00:06 2022] DELETE /identity/v3/projects/c3f84feae23345efbeea2f244472d5fa => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:00:06.797448 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b1dd8-7ec0-46ec-8a97-30efae949e72 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:06.797731 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b1dd8-7ec0-46ec-8a97-30efae949e72 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:06.798004 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b1dd8-7ec0-46ec-8a97-30efae949e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:06.827641 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-4b7b1dd8-7ec0-46ec-8a97-30efae949e72 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:06.828947 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-4b7b1dd8-7ec0-46ec-8a97-30efae949e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:06.904149 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1424/2812] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:06.912044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba2f81-06ae-43ed-8442-d5bda59daaf8 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:06.912517 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba2f81-06ae-43ed-8442-d5bda59daaf8 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:06.912657 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba2f81-06ae-43ed-8442-d5bda59daaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:06.933418 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-f1ba2f81-06ae-43ed-8442-d5bda59daaf8 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:06.934669 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-f1ba2f81-06ae-43ed-8442-d5bda59daaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:06.945849 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1389/2813] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:06.950037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b1dd8-7ec0-46ec-8a97-30efae949e72 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:06.960375 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b1dd8-7ec0-46ec-8a97-30efae949e72 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:06.967014 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b7e09-3880-4552-b82c-4c47264d5492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:06.967742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b7e09-3880-4552-b82c-4c47264d5492 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:06.968052 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b7e09-3880-4552-b82c-4c47264d5492 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:06.968328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b7e09-3880-4552-b82c-4c47264d5492 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:06.969073 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b7e09-3880-4552-b82c-4c47264d5492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 15:00:06.969782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b7e09-3880-4552-b82c-4c47264d5492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:06.971853 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b7e09-3880-4552-b82c-4c47264d5492 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:06.986192 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1425/2814] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 15:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 15:00:06.990601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba2f81-06ae-43ed-8442-d5bda59daaf8 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:07.008728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba2f81-06ae-43ed-8442-d5bda59daaf8 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:07.016365 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb709cd5-d445-497f-9e84-803bb55bbf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:07.017138 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb709cd5-d445-497f-9e84-803bb55bbf5c admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:07.017542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb709cd5-d445-497f-9e84-803bb55bbf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:07.017942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb709cd5-d445-497f-9e84-803bb55bbf5c admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:07.018720 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb709cd5-d445-497f-9e84-803bb55bbf5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:07.020895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb709cd5-d445-497f-9e84-803bb55bbf5c admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:07.049741 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1390/2815] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 15:00:06 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 15:00:07.054609 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b7e09-3880-4552-b82c-4c47264d5492 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:07.064649 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b7e09-3880-4552-b82c-4c47264d5492 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:07.122956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fff301-5c22-45c9-a98f-ea737bb46524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:07.123582 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fff301-5c22-45c9-a98f-ea737bb46524 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:07.123881 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fff301-5c22-45c9-a98f-ea737bb46524 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:07.124149 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fff301-5c22-45c9-a98f-ea737bb46524 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:07.124834 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fff301-5c22-45c9-a98f-ea737bb46524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:07.126702 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fff301-5c22-45c9-a98f-ea737bb46524 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:07.149465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1426/2816] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 15:00:07 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 15:00:07.153678 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb709cd5-d445-497f-9e84-803bb55bbf5c admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:07.171655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb709cd5-d445-497f-9e84-803bb55bbf5c admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:07.180486 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289cc72-b4fe-4c27-bff3-331dccc91f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:07.181111 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289cc72-b4fe-4c27-bff3-331dccc91f88 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:07.181402 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289cc72-b4fe-4c27-bff3-331dccc91f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:07.181648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289cc72-b4fe-4c27-bff3-331dccc91f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:07.183102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289cc72-b4fe-4c27-bff3-331dccc91f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:07.185050 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289cc72-b4fe-4c27-bff3-331dccc91f88 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:07.197434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1391/2817] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 15:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 15:00:07.201285 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fff301-5c22-45c9-a98f-ea737bb46524 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:07.212545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fff301-5c22-45c9-a98f-ea737bb46524 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:07.217531 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ba5de-1249-4420-b992-d514dca90489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:07.218269 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ba5de-1249-4420-b992-d514dca90489 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:07.218660 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ba5de-1249-4420-b992-d514dca90489 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:07.218660 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ba5de-1249-4420-b992-d514dca90489 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:07.219314 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ba5de-1249-4420-b992-d514dca90489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:07.221120 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ba5de-1249-4420-b992-d514dca90489 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:07.232961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1427/2818] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 15:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 15:00:07.237522 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289cc72-b4fe-4c27-bff3-331dccc91f88 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:07.251960 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289cc72-b4fe-4c27-bff3-331dccc91f88 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:07.258190 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c8c42-1b75-4a17-9064-cb689a1c67c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:07.260043 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c8c42-1b75-4a17-9064-cb689a1c67c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:07.260470 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c8c42-1b75-4a17-9064-cb689a1c67c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:07.260752 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c8c42-1b75-4a17-9064-cb689a1c67c9 admin admin] PATH_INFO: `/v3/projects/302943a30be24db88755d3e03a0491a9/users/3c05801b5bce4e4890ae0dd4ae67b632/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:07.279296 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c8c42-1b75-4a17-9064-cb689a1c67c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=302943a30be24db88755d3e03a0491a9, user_id=3c05801b5bce4e4890ae0dd4ae67b632, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:07.282251 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c8c42-1b75-4a17-9064-cb689a1c67c9 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:07.296192 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1392/2819] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 15:00:07 2022] PUT /identity/v3/projects/302943a30be24db88755d3e03a0491a9/users/3c05801b5bce4e4890ae0dd4ae67b632/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 15:00:07.308053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc83c4d-9494-4256-ade3-c61f08b6be93 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:07.308866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc83c4d-9494-4256-ade3-c61f08b6be93 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:07.308866 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc83c4d-9494-4256-ade3-c61f08b6be93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:07.357394 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-3fc83c4d-9494-4256-ade3-c61f08b6be93 None None] MFA Rules not processed for user `3c05801b5bce4e4890ae0dd4ae67b632`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:07.358818 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-3fc83c4d-9494-4256-ade3-c61f08b6be93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:07.416727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:07.440877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1428/2820] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 15:00:07.453523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c8c42-1b75-4a17-9064-cb689a1c67c9 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:07.455487 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-674c8c42-1b75-4a17-9064-cb689a1c67c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:07.531047 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c8c42-1b75-4a17-9064-cb689a1c67c9 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:07.538393 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73955d4c-8906-486f-b6f7-8ec869cd2e4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:07.539035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73955d4c-8906-486f-b6f7-8ec869cd2e4f service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:07.539713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73955d4c-8906-486f-b6f7-8ec869cd2e4f service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:07.539713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73955d4c-8906-486f-b6f7-8ec869cd2e4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:07.554036 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73955d4c-8906-486f-b6f7-8ec869cd2e4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:07.556076 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73955d4c-8906-486f-b6f7-8ec869cd2e4f service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:07.570035 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1393/2821] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:00:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:00:10.206179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc83c4d-9494-4256-ade3-c61f08b6be93 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:10.271593 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc83c4d-9494-4256-ade3-c61f08b6be93 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:10.276791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:10.277465 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:10.277740 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:10.278012 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] PATH_INFO: `/v3/users/3c05801b5bce4e4890ae0dd4ae67b632` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:10.279768 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c05801b5bce4e4890ae0dd4ae67b632)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:10.281089 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:10.326233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c05801b5bce4e4890ae0dd4ae67b632'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:10.333068 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c05801b5bce4e4890ae0dd4ae67b632'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:10.333726 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c05801b5bce4e4890ae0dd4ae67b632'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:10.347438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c05801b5bce4e4890ae0dd4ae67b632'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:10.357862 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c05801b5bce4e4890ae0dd4ae67b632'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:10.365154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1429/2822] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 15:00:10 2022] DELETE /identity/v3/users/3c05801b5bce4e4890ae0dd4ae67b632 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:00:10.369726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb546bf-abc7-462d-a519-e9e8fbbfbf63 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:10.370194 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb546bf-abc7-462d-a519-e9e8fbbfbf63 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:10.370332 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb546bf-abc7-462d-a519-e9e8fbbfbf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:10.389519 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-beb546bf-abc7-462d-a519-e9e8fbbfbf63 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:10.390994 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-beb546bf-abc7-462d-a519-e9e8fbbfbf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:10.421981 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:10.454053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1394/2823] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:10.464567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:10.466200 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:10.526844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bad51-ec1a-40d6-b8dc-24ada8f456d1 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:10.530426 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561876a5-48bc-47b7-874b-3bfe191da720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:10.531022 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561876a5-48bc-47b7-874b-3bfe191da720 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:10.531298 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561876a5-48bc-47b7-874b-3bfe191da720 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:10.531567 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561876a5-48bc-47b7-874b-3bfe191da720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:10.540488 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561876a5-48bc-47b7-874b-3bfe191da720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:10.541923 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561876a5-48bc-47b7-874b-3bfe191da720 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:10.551180 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1430/2824] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:00:10.828613 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb546bf-abc7-462d-a519-e9e8fbbfbf63 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:10.830088 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-beb546bf-abc7-462d-a519-e9e8fbbfbf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:10.842756 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb546bf-abc7-462d-a519-e9e8fbbfbf63 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:10.856615 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e967b-8344-4a39-8704-007575a364b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:10.857731 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e967b-8344-4a39-8704-007575a364b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:10.858034 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e967b-8344-4a39-8704-007575a364b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:10.858370 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e967b-8344-4a39-8704-007575a364b0 admin admin] PATH_INFO: `/v3/projects/302943a30be24db88755d3e03a0491a9` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:10.859878 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e967b-8344-4a39-8704-007575a364b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=302943a30be24db88755d3e03a0491a9)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:10.861818 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e967b-8344-4a39-8704-007575a364b0 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:10.912857 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-283e967b-8344-4a39-8704-007575a364b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '302943a30be24db88755d3e03a0491a9', 'request_id': 'req-283e967b-8344-4a39-8704-007575a364b0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:10.919620 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-283e967b-8344-4a39-8704-007575a364b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '302943a30be24db88755d3e03a0491a9', 'request_id': 'req-283e967b-8344-4a39-8704-007575a364b0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:10.928846 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-283e967b-8344-4a39-8704-007575a364b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '302943a30be24db88755d3e03a0491a9', 'request_id': 'req-283e967b-8344-4a39-8704-007575a364b0'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:10.940388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-283e967b-8344-4a39-8704-007575a364b0 admin admin] The token cache is being invalidate because project 302943a30be24db88755d3e03a0491a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 15:00:10.940695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-283e967b-8344-4a39-8704-007575a364b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:10.941325 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-283e967b-8344-4a39-8704-007575a364b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:10.942843 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1395/2825] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 15:00:10 2022] DELETE /identity/v3/projects/302943a30be24db88755d3e03a0491a9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:00:10.948520 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b2a5e-df9d-4d06-ad94-3d553163336a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:10.948885 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b2a5e-df9d-4d06-ad94-3d553163336a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:10.949117 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b2a5e-df9d-4d06-ad94-3d553163336a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:10.956812 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:10.972040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-5c5b2a5e-df9d-4d06-ad94-3d553163336a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:10.973346 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5c5b2a5e-df9d-4d06-ad94-3d553163336a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:11.032700 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1431/2826] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:11.041065 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a048a7-5e38-4bb4-bccf-fc2d11dd795a None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:11.041349 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a048a7-5e38-4bb4-bccf-fc2d11dd795a None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:11.041648 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a048a7-5e38-4bb4-bccf-fc2d11dd795a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:11.062672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-40a048a7-5e38-4bb4-bccf-fc2d11dd795a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:11.063899 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-40a048a7-5e38-4bb4-bccf-fc2d11dd795a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:11.074714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1396/2827] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:11.079638 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b2a5e-df9d-4d06-ad94-3d553163336a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:11.090522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b2a5e-df9d-4d06-ad94-3d553163336a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:11.095036 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e40ee-4437-4132-8858-c93a0893bb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:11.095631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e40ee-4437-4132-8858-c93a0893bb61 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:11.095941 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e40ee-4437-4132-8858-c93a0893bb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:11.096215 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e40ee-4437-4132-8858-c93a0893bb61 admin admin] PATH_INFO: `/v3/domains` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:11.096877 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e40ee-4437-4132-8858-c93a0893bb61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 15:00:11.097179 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e40ee-4437-4132-8858-c93a0893bb61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:11.098770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e40ee-4437-4132-8858-c93a0893bb61 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:11.110742 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1432/2828] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 15:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 15:00:11.115190 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a048a7-5e38-4bb4-bccf-fc2d11dd795a None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:11.124235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a048a7-5e38-4bb4-bccf-fc2d11dd795a None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:11.128969 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bf707-4689-4664-8023-5572b68cccf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:11.129557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bf707-4689-4664-8023-5572b68cccf2 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:11.129833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bf707-4689-4664-8023-5572b68cccf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:11.130102 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bf707-4689-4664-8023-5572b68cccf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:11.130725 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bf707-4689-4664-8023-5572b68cccf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:11.132691 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bf707-4689-4664-8023-5572b68cccf2 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:11.165481 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1397/2829] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 15:00:11 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 15:00:11.166711 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e40ee-4437-4132-8858-c93a0893bb61 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:11.177288 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e40ee-4437-4132-8858-c93a0893bb61 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:11.234049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b10bb1-dc02-4e6e-b750-de8fae470737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:11.234707 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b10bb1-dc02-4e6e-b750-de8fae470737 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:11.234812 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b10bb1-dc02-4e6e-b750-de8fae470737 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:11.235079 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b10bb1-dc02-4e6e-b750-de8fae470737 admin admin] PATH_INFO: `/v3/users` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:11.235707 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b10bb1-dc02-4e6e-b750-de8fae470737 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:11.237617 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b10bb1-dc02-4e6e-b750-de8fae470737 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:11.258959 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1433/2830] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 15:00:11 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 15:00:11.263048 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bf707-4689-4664-8023-5572b68cccf2 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:11.273490 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bf707-4689-4664-8023-5572b68cccf2 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:11.278084 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5e3dc-a830-4ce1-bf12-702f5ce88f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:11.278670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5e3dc-a830-4ce1-bf12-702f5ce88f05 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:11.278943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5e3dc-a830-4ce1-bf12-702f5ce88f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:11.279210 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5e3dc-a830-4ce1-bf12-702f5ce88f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:11.279868 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5e3dc-a830-4ce1-bf12-702f5ce88f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:11.281532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5e3dc-a830-4ce1-bf12-702f5ce88f05 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:11.292137 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1398/2831] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 15:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 15:00:11.295967 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b10bb1-dc02-4e6e-b750-de8fae470737 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:11.306498 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b10bb1-dc02-4e6e-b750-de8fae470737 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:11.311086 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46861d-7637-4479-861b-2fd85734da2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:11.311670 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46861d-7637-4479-861b-2fd85734da2d admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:11.311981 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46861d-7637-4479-861b-2fd85734da2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:11.312255 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46861d-7637-4479-861b-2fd85734da2d admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:11.312890 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46861d-7637-4479-861b-2fd85734da2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:11.314507 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46861d-7637-4479-861b-2fd85734da2d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:11.324932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1434/2832] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 15:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 15:00:11.328933 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5e3dc-a830-4ce1-bf12-702f5ce88f05 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:11.338705 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5e3dc-a830-4ce1-bf12-702f5ce88f05 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:11.343328 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33f100-8177-459f-8262-927106b3fe0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:11.344049 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33f100-8177-459f-8262-927106b3fe0e admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:11.344329 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33f100-8177-459f-8262-927106b3fe0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:11.344600 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33f100-8177-459f-8262-927106b3fe0e admin admin] PATH_INFO: `/v3/projects/4ee37b6e59d646fbbb392698cc864664/users/d12a394a0160414db686644696d243d6/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:11.360180 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33f100-8177-459f-8262-927106b3fe0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee37b6e59d646fbbb392698cc864664, user_id=d12a394a0160414db686644696d243d6, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:11.362829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33f100-8177-459f-8262-927106b3fe0e admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:11.376943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1399/2833] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 15:00:11 2022] PUT /identity/v3/projects/4ee37b6e59d646fbbb392698cc864664/users/d12a394a0160414db686644696d243d6/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:00:11.388854 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3beb9-0750-4e07-af93-2fc8d9004428 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:11.389137 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3beb9-0750-4e07-af93-2fc8d9004428 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:11.389409 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3beb9-0750-4e07-af93-2fc8d9004428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:11.417825 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-d0c3beb9-0750-4e07-af93-2fc8d9004428 None None] MFA Rules not processed for user `d12a394a0160414db686644696d243d6`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:11.419070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d0c3beb9-0750-4e07-af93-2fc8d9004428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:11.477908 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:11.484309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1435/2834] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:11.496636 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33f100-8177-459f-8262-927106b3fe0e admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:11.498387 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-7c33f100-8177-459f-8262-927106b3fe0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:11.555352 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33f100-8177-459f-8262-927106b3fe0e admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:11.559100 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae041ce-b57d-4ab7-a849-48d581e345aa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f2b6dc3406f46e68c74677f5fc7e3b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:11.559728 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae041ce-b57d-4ab7-a849-48d581e345aa service glance] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:11.560006 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae041ce-b57d-4ab7-a849-48d581e345aa service glance] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:11.560274 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae041ce-b57d-4ab7-a849-48d581e345aa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:11.566637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:11.569770 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae041ce-b57d-4ab7-a849-48d581e345aa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:11.571541 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae041ce-b57d-4ab7-a849-48d581e345aa service glance] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:11.582603 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1400/2835] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:00:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:00:21.484823 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:21.583432 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=2 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:37.235388 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3beb9-0750-4e07-af93-2fc8d9004428 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:37.237467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-d0c3beb9-0750-4e07-af93-2fc8d9004428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:37.297938 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3beb9-0750-4e07-af93-2fc8d9004428 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:37.301648 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e04231-9c2c-4341-8508-5350a0988d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:37.302255 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e04231-9c2c-4341-8508-5350a0988d10 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:37.302524 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e04231-9c2c-4341-8508-5350a0988d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:37.302797 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e04231-9c2c-4341-8508-5350a0988d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:37.305250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-55e04231-9c2c-4341-8508-5350a0988d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:37.365210 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e04231-9c2c-4341-8508-5350a0988d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:37.367147 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e04231-9c2c-4341-8508-5350a0988d10 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:37.376422 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1436/2836] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:00:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:00:39.420413 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae041ce-b57d-4ab7-a849-48d581e345aa service glance] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:39.493132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae041ce-b57d-4ab7-a849-48d581e345aa service glance] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:39.499416 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:39.500118 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:39.500396 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:39.500831 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] PATH_INFO: `/v3/users/d12a394a0160414db686644696d243d6` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:39.502063 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d12a394a0160414db686644696d243d6)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:39.504614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:39.562895 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd12a394a0160414db686644696d243d6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:39.572115 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd12a394a0160414db686644696d243d6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:39.572997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd12a394a0160414db686644696d243d6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:39.573870 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd12a394a0160414db686644696d243d6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:39.589869 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd12a394a0160414db686644696d243d6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:39.609078 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1401/2837] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 15:00:39 2022] DELETE /identity/v3/users/d12a394a0160414db686644696d243d6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:00:39.610969 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42402945-f22e-41e5-a8bf-1a4b8d9e059a None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:39.611459 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42402945-f22e-41e5-a8bf-1a4b8d9e059a None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:39.611565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42402945-f22e-41e5-a8bf-1a4b8d9e059a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:39.634585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-42402945-f22e-41e5-a8bf-1a4b8d9e059a None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:39.635874 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-42402945-f22e-41e5-a8bf-1a4b8d9e059a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:39.708278 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1437/2838] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:39.718700 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:39.720301 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:39.780714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e587d03-367b-401e-a8e0-2d84170895b1 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:39.784365 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8dd11-86f0-4d14-a872-d9a2614b32cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:39.784975 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8dd11-86f0-4d14-a872-d9a2614b32cf service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:39.785250 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8dd11-86f0-4d14-a872-d9a2614b32cf service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:39.785722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8dd11-86f0-4d14-a872-d9a2614b32cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:39.796609 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8dd11-86f0-4d14-a872-d9a2614b32cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:39.798901 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8dd11-86f0-4d14-a872-d9a2614b32cf service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:39.810829 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1402/2839] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:00:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:00:40.104641 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42402945-f22e-41e5-a8bf-1a4b8d9e059a None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:40.106076 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-42402945-f22e-41e5-a8bf-1a4b8d9e059a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:40.120324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42402945-f22e-41e5-a8bf-1a4b8d9e059a None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:40.125088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762149c7-06dc-46df-b183-b545fcc42080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:40.126281 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762149c7-06dc-46df-b183-b545fcc42080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:40.126721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762149c7-06dc-46df-b183-b545fcc42080 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:40.126999 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762149c7-06dc-46df-b183-b545fcc42080 admin admin] PATH_INFO: `/v3/projects/4ee37b6e59d646fbbb392698cc864664` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:40.129109 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762149c7-06dc-46df-b183-b545fcc42080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee37b6e59d646fbbb392698cc864664)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:40.131725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762149c7-06dc-46df-b183-b545fcc42080 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:40.181232 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:40.184730 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-762149c7-06dc-46df-b183-b545fcc42080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee37b6e59d646fbbb392698cc864664', 'request_id': 'req-762149c7-06dc-46df-b183-b545fcc42080'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:40.191172 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-762149c7-06dc-46df-b183-b545fcc42080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee37b6e59d646fbbb392698cc864664', 'request_id': 'req-762149c7-06dc-46df-b183-b545fcc42080'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:40.198579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-762149c7-06dc-46df-b183-b545fcc42080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee37b6e59d646fbbb392698cc864664', 'request_id': 'req-762149c7-06dc-46df-b183-b545fcc42080'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:40.209573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-762149c7-06dc-46df-b183-b545fcc42080 admin admin] The token cache is being invalidate because project 4ee37b6e59d646fbbb392698cc864664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 15:00:40.209858 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-762149c7-06dc-46df-b183-b545fcc42080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:40.210517 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-762149c7-06dc-46df-b183-b545fcc42080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:00:40.212007 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1438/2840] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 15:00:40 2022] DELETE /identity/v3/projects/4ee37b6e59d646fbbb392698cc864664 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:00:40.216836 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4b99d-7cdd-4bbd-b742-26b70d2bda50 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:40.217117 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4b99d-7cdd-4bbd-b742-26b70d2bda50 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:40.217390 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4b99d-7cdd-4bbd-b742-26b70d2bda50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:40.236575 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-1bd4b99d-7cdd-4bbd-b742-26b70d2bda50 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:40.237763 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1bd4b99d-7cdd-4bbd-b742-26b70d2bda50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:40.254438 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:40.298517 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1403/2841] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:40.306390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a035f4-305e-4b26-83f8-a65668a8e02b None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:40.306680 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a035f4-305e-4b26-83f8-a65668a8e02b None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:40.306956 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a035f4-305e-4b26-83f8-a65668a8e02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:40.326875 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-c4a035f4-305e-4b26-83f8-a65668a8e02b None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:40.328131 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c4a035f4-305e-4b26-83f8-a65668a8e02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:40.336512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1439/2842] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:40.340928 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4b99d-7cdd-4bbd-b742-26b70d2bda50 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:40.351356 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4b99d-7cdd-4bbd-b742-26b70d2bda50 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:40.356230 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ca1ee-a2f1-4db8-b462-9db646a516a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:40.356804 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ca1ee-a2f1-4db8-b462-9db646a516a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:40.357080 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ca1ee-a2f1-4db8-b462-9db646a516a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:40.357353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ca1ee-a2f1-4db8-b462-9db646a516a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:40.357991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ca1ee-a2f1-4db8-b462-9db646a516a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93179) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 15:00:40.358293 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ca1ee-a2f1-4db8-b462-9db646a516a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:40.360333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ca1ee-a2f1-4db8-b462-9db646a516a3 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:40.372081 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1404/2843] 10.222.0.36 () {58 vars in 1153 bytes} [Thu Aug 18 15:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 15:00:40.376342 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a035f4-305e-4b26-83f8-a65668a8e02b None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:40.385585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a035f4-305e-4b26-83f8-a65668a8e02b None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:40.390354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6627e-ccba-4c25-b432-24f76cc6a9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:40.391205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6627e-ccba-4c25-b432-24f76cc6a9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:40.391481 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6627e-ccba-4c25-b432-24f76cc6a9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:40.391779 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6627e-ccba-4c25-b432-24f76cc6a9f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:40.392403 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6627e-ccba-4c25-b432-24f76cc6a9f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:40.394292 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6627e-ccba-4c25-b432-24f76cc6a9f7 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:40.421916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1440/2844] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 15:00:40 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 15:00:40.426053 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ca1ee-a2f1-4db8-b462-9db646a516a3 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:40.436070 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ca1ee-a2f1-4db8-b462-9db646a516a3 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:40.492850 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310e2aa-d05f-4436-90f2-790f39d96d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:40.493977 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310e2aa-d05f-4436-90f2-790f39d96d81 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:40.494495 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310e2aa-d05f-4436-90f2-790f39d96d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:40.494782 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310e2aa-d05f-4436-90f2-790f39d96d81 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:40.495751 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310e2aa-d05f-4436-90f2-790f39d96d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:40.497865 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310e2aa-d05f-4436-90f2-790f39d96d81 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:40.519450 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1405/2845] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 15:00:40 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 15:00:40.523800 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6627e-ccba-4c25-b432-24f76cc6a9f7 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:40.534719 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6627e-ccba-4c25-b432-24f76cc6a9f7 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:40.539529 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2277e4-9ffe-4938-8cb8-8a67c9cb17df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:40.540203 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2277e4-9ffe-4938-8cb8-8a67c9cb17df admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:40.540476 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2277e4-9ffe-4938-8cb8-8a67c9cb17df admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:40.540830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2277e4-9ffe-4938-8cb8-8a67c9cb17df admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:40.541635 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2277e4-9ffe-4938-8cb8-8a67c9cb17df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:40.543202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2277e4-9ffe-4938-8cb8-8a67c9cb17df admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:40.553834 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1441/2846] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 15:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 15:00:40.557710 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310e2aa-d05f-4436-90f2-790f39d96d81 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:40.567646 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310e2aa-d05f-4436-90f2-790f39d96d81 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:40.572798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bfa36b-8f71-4059-911d-1cf7c6b1cd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:40.573386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfa36b-8f71-4059-911d-1cf7c6b1cd64 admin admin] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:40.573671 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfa36b-8f71-4059-911d-1cf7c6b1cd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:40.573948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfa36b-8f71-4059-911d-1cf7c6b1cd64 admin admin] PATH_INFO: `/v3/roles` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:40.574581 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bfa36b-8f71-4059-911d-1cf7c6b1cd64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:40.576217 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bfa36b-8f71-4059-911d-1cf7c6b1cd64 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:40.587218 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1406/2847] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 15:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 15:00:40.591299 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2277e4-9ffe-4938-8cb8-8a67c9cb17df admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:40.601488 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2277e4-9ffe-4938-8cb8-8a67c9cb17df admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:40.606055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c6d08-faeb-41d0-8c66-cecb99cca7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:40.606690 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c6d08-faeb-41d0-8c66-cecb99cca7e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:40.606964 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c6d08-faeb-41d0-8c66-cecb99cca7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:40.607233 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c6d08-faeb-41d0-8c66-cecb99cca7e5 admin admin] PATH_INFO: `/v3/projects/1f169db98a4549109d83c18d108b8282/users/57632501742348109ed397d84c71510a/roles/603573d2738b49c389ad913208d2ced1` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:40.621793 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c6d08-faeb-41d0-8c66-cecb99cca7e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f169db98a4549109d83c18d108b8282, user_id=57632501742348109ed397d84c71510a, role_id=603573d2738b49c389ad913208d2ced1)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:40.624651 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c6d08-faeb-41d0-8c66-cecb99cca7e5 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:40.637657 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1442/2848] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 15:00:40 2022] PUT /identity/v3/projects/1f169db98a4549109d83c18d108b8282/users/57632501742348109ed397d84c71510a/roles/603573d2738b49c389ad913208d2ced1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 15:00:40.642862 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164d1dc-58ca-488c-87e3-09259503ab90 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:40.643142 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164d1dc-58ca-488c-87e3-09259503ab90 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:40.643412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164d1dc-58ca-488c-87e3-09259503ab90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:40.662353 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-8164d1dc-58ca-488c-87e3-09259503ab90 None None] MFA Rules not processed for user `5064296cd28f403599a02675cb0c7103`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:40.664132 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8164d1dc-58ca-488c-87e3-09259503ab90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:40.722461 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1407/2849] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:40.732575 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c6d08-faeb-41d0-8c66-cecb99cca7e5 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:40.734283 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-830c6d08-faeb-41d0-8c66-cecb99cca7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:40.793782 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c6d08-faeb-41d0-8c66-cecb99cca7e5 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:40.797412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c530125-3411-4e0c-a03b-8d3968952488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:40.798003 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c530125-3411-4e0c-a03b-8d3968952488 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:40.798277 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c530125-3411-4e0c-a03b-8d3968952488 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:40.798546 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c530125-3411-4e0c-a03b-8d3968952488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:40.807920 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c530125-3411-4e0c-a03b-8d3968952488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:40.809339 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c530125-3411-4e0c-a03b-8d3968952488 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:40.818599 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1443/2850] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:00:44.811114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791b901-8b18-4a87-b7de-5a82f3d0b5ed None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:44.811672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791b901-8b18-4a87-b7de-5a82f3d0b5ed None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:44.811807 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791b901-8b18-4a87-b7de-5a82f3d0b5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:44.841792 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-3791b901-8b18-4a87-b7de-5a82f3d0b5ed None None] MFA Rules not processed for user `57632501742348109ed397d84c71510a`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:44.843028 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3791b901-8b18-4a87-b7de-5a82f3d0b5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:44.861436 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:44.920060 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1408/2851] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:44.928107 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c530125-3411-4e0c-a03b-8d3968952488 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:44.941146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c530125-3411-4e0c-a03b-8d3968952488 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:44.946358 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b598ae-a381-462d-8339-442cc3bc5c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:44.947055 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b598ae-a381-462d-8339-442cc3bc5c13 admin admin] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:44.947404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b598ae-a381-462d-8339-442cc3bc5c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:44.947767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b598ae-a381-462d-8339-442cc3bc5c13 admin admin] PATH_INFO: `/v3/projects` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:44.948480 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b598ae-a381-462d-8339-442cc3bc5c13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:44.950573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b598ae-a381-462d-8339-442cc3bc5c13 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:44.980573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1444/2852] 10.222.0.36 () {60 vars in 1153 bytes} [Thu Aug 18 15:00:44 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 15:00:44.985195 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791b901-8b18-4a87-b7de-5a82f3d0b5ed None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:44.996730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791b901-8b18-4a87-b7de-5a82f3d0b5ed None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:45.058673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16eef76-da80-4b4d-be88-7d9720e6c434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:45.059405 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eef76-da80-4b4d-be88-7d9720e6c434 admin admin] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:45.059785 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eef76-da80-4b4d-be88-7d9720e6c434 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:45.060131 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eef76-da80-4b4d-be88-7d9720e6c434 admin admin] PATH_INFO: `/v3/users` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:45.060868 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16eef76-da80-4b4d-be88-7d9720e6c434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:45.062898 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16eef76-da80-4b4d-be88-7d9720e6c434 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:45.085716 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1409/2853] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Aug 18 15:00:44 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 15:00:45.089727 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b598ae-a381-462d-8339-442cc3bc5c13 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:45.101118 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b598ae-a381-462d-8339-442cc3bc5c13 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:45.106041 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138eef49-8c64-44e2-ab5a-e0cd8da81eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:45.106643 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138eef49-8c64-44e2-ab5a-e0cd8da81eab admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:45.106916 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138eef49-8c64-44e2-ab5a-e0cd8da81eab admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:45.107242 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138eef49-8c64-44e2-ab5a-e0cd8da81eab admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:45.107917 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138eef49-8c64-44e2-ab5a-e0cd8da81eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:45.109519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138eef49-8c64-44e2-ab5a-e0cd8da81eab admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:45.121356 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1445/2854] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 15:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 15:00:45.125616 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16eef76-da80-4b4d-be88-7d9720e6c434 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:45.135786 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16eef76-da80-4b4d-be88-7d9720e6c434 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:45.140880 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23cd7dc-8722-44e9-aecc-c7c3c3428ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:45.141542 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cd7dc-8722-44e9-aecc-c7c3c3428ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:45.141816 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cd7dc-8722-44e9-aecc-c7c3c3428ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:45.142235 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cd7dc-8722-44e9-aecc-c7c3c3428ccc admin admin] PATH_INFO: `/v3/projects/59bf5e892b4043db9628d6f3b4149aab/users/8951ef845ca74c2cb4df80d364534c6d/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:45.158532 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cd7dc-8722-44e9-aecc-c7c3c3428ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bf5e892b4043db9628d6f3b4149aab, user_id=8951ef845ca74c2cb4df80d364534c6d, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:45.161355 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cd7dc-8722-44e9-aecc-c7c3c3428ccc admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:45.175945 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1410/2855] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 15:00:45 2022] PUT /identity/v3/projects/59bf5e892b4043db9628d6f3b4149aab/users/8951ef845ca74c2cb4df80d364534c6d/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 15:00:45.180542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138eef49-8c64-44e2-ab5a-e0cd8da81eab admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:45.191438 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138eef49-8c64-44e2-ab5a-e0cd8da81eab admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:45.230631 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:45.252429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb43a1-e1e3-41b8-b564-8607d74710f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:45.253070 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb43a1-e1e3-41b8-b564-8607d74710f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:45.253351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb43a1-e1e3-41b8-b564-8607d74710f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:45.253622 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb43a1-e1e3-41b8-b564-8607d74710f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:45.254462 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb43a1-e1e3-41b8-b564-8607d74710f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:45.255903 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb43a1-e1e3-41b8-b564-8607d74710f5 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:45.267088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1446/2856] 10.222.0.36 () {58 vars in 1122 bytes} [Thu Aug 18 15:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 15:00:45.271296 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23cd7dc-8722-44e9-aecc-c7c3c3428ccc admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:45.282243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23cd7dc-8722-44e9-aecc-c7c3c3428ccc admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:45.286865 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29ff00-c9fd-499a-87e9-f53171bafa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:45.287510 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29ff00-c9fd-499a-87e9-f53171bafa15 admin admin] REQUEST_METHOD: `PUT` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:45.287833 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29ff00-c9fd-499a-87e9-f53171bafa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:45.288133 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29ff00-c9fd-499a-87e9-f53171bafa15 admin admin] PATH_INFO: `/v3/projects/59bf5e892b4043db9628d6f3b4149aab/users/8951ef845ca74c2cb4df80d364534c6d/roles/9a439cf8eef34ada85428b378878e63e` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:45.290050 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29ff00-c9fd-499a-87e9-f53171bafa15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bf5e892b4043db9628d6f3b4149aab, user_id=8951ef845ca74c2cb4df80d364534c6d, role_id=9a439cf8eef34ada85428b378878e63e)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:45.292732 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29ff00-c9fd-499a-87e9-f53171bafa15 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:45.302810 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1411/2857] 10.222.0.36 () {60 vars in 1483 bytes} [Thu Aug 18 15:00:45 2022] PUT /identity/v3/projects/59bf5e892b4043db9628d6f3b4149aab/users/8951ef845ca74c2cb4df80d364534c6d/roles/9a439cf8eef34ada85428b378878e63e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:00:49.783980 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186416a8-1eb2-4b37-a420-a9792d5e0877 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:49.784542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186416a8-1eb2-4b37-a420-a9792d5e0877 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:49.784542 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186416a8-1eb2-4b37-a420-a9792d5e0877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:49.815310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-186416a8-1eb2-4b37-a420-a9792d5e0877 None None] MFA Rules not processed for user `8951ef845ca74c2cb4df80d364534c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:49.816589 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-186416a8-1eb2-4b37-a420-a9792d5e0877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:49.903703 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1447/2858] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:49.915249 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29ff00-c9fd-499a-87e9-f53171bafa15 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:49.917045 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-ff29ff00-c9fd-499a-87e9-f53171bafa15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:49.983906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29ff00-c9fd-499a-87e9-f53171bafa15 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:49.987601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c78684-4b8d-440f-9036-0914c78b7f90 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f2b6dc3406f46e68c74677f5fc7e3b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:49.988243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c78684-4b8d-440f-9036-0914c78b7f90 service glance] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:49.988527 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c78684-4b8d-440f-9036-0914c78b7f90 service glance] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:49.988802 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c78684-4b8d-440f-9036-0914c78b7f90 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:50.015375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:00:50.050151 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c78684-4b8d-440f-9036-0914c78b7f90 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:50.051590 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c78684-4b8d-440f-9036-0914c78b7f90 service glance] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:50.061457 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1412/2859] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:00:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:00:50.192836 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae083de7-2a48-4b46-a2f7-556e7033e6cd None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:50.193127 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae083de7-2a48-4b46-a2f7-556e7033e6cd None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:50.193404 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae083de7-2a48-4b46-a2f7-556e7033e6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:50.227311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-ae083de7-2a48-4b46-a2f7-556e7033e6cd None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:50.228456 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-ae083de7-2a48-4b46-a2f7-556e7033e6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:50.290512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1448/2860] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 15:00:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:00:50.354882 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c78684-4b8d-440f-9036-0914c78b7f90 service glance] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:00:50.357113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-50c78684-4b8d-440f-9036-0914c78b7f90 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:50.442422 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c78684-4b8d-440f-9036-0914c78b7f90 service glance] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:00:50.447740 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34cc749-3613-4d5d-97b8-83629aeb237f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:00:50.448680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34cc749-3613-4d5d-97b8-83629aeb237f service swift] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:50.449019 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34cc749-3613-4d5d-97b8-83629aeb237f service swift] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:50.449348 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34cc749-3613-4d5d-97b8-83629aeb237f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:50.470089 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34cc749-3613-4d5d-97b8-83629aeb237f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:00:50.472014 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34cc749-3613-4d5d-97b8-83629aeb237f service swift] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:00:50.481423 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1413/2861] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 15:00:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 15:00:58.955252 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7648da9-088c-40b2-a9af-356a65b80d75 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:00:58.955844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7648da9-088c-40b2-a9af-356a65b80d75 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:00:58.955844 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7648da9-088c-40b2-a9af-356a65b80d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:00:58.975059 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-a7648da9-088c-40b2-a9af-356a65b80d75 None None] MFA Rules not processed for user `57632501742348109ed397d84c71510a`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:00:58.976830 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a7648da9-088c-40b2-a9af-356a65b80d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:00:58.983647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1449/2862] 10.222.0.36 () {58 vars in 958 bytes} [Thu Aug 18 15:00:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:01:00.477726 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=48 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:01:01.213954 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41d6b6-b405-469e-b470-a231f421ffff None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:01:01.214333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41d6b6-b405-469e-b470-a231f421ffff None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:01:01.214611 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41d6b6-b405-469e-b470-a231f421ffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:01:01.234723 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-bd41d6b6-b405-469e-b470-a231f421ffff None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:01:01.235985 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bd41d6b6-b405-469e-b470-a231f421ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:01:01.243625 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1414/2863] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Aug 18 15:01:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:01:01.255594 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7648da9-088c-40b2-a9af-356a65b80d75 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:01:01.268468 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7648da9-088c-40b2-a9af-356a65b80d75 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:01:01.273328 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ac05d-52a4-4d37-911c-a109453d9385 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:01:01.273922 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ac05d-52a4-4d37-911c-a109453d9385 service swift] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:01:01.274205 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ac05d-52a4-4d37-911c-a109453d9385 service swift] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:01:01.274479 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ac05d-52a4-4d37-911c-a109453d9385 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:01:01.282620 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ac05d-52a4-4d37-911c-a109453d9385 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:01:01.284228 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ac05d-52a4-4d37-911c-a109453d9385 service swift] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:01:01.292660 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1450/2864] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 15:01:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 15:01:11.229375 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=3 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:01:11.289552 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=83 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:01:20.783020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41d6b6-b405-469e-b470-a231f421ffff None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:01:20.784456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-bd41d6b6-b405-469e-b470-a231f421ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:01:20.839759 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41d6b6-b405-469e-b470-a231f421ffff None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:01:20.843245 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ad3f4-aa7e-4332-bdf3-e5418eb842e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9b29546bc848ef9e92b3d267da18ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:01:20.843884 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ad3f4-aa7e-4332-bdf3-e5418eb842e7 service cinder] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:01:20.844163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ad3f4-aa7e-4332-bdf3-e5418eb842e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:01:20.844434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ad3f4-aa7e-4332-bdf3-e5418eb842e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:01:20.846177 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9a4ad3f4-aa7e-4332-bdf3-e5418eb842e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:01:20.918811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ad3f4-aa7e-4332-bdf3-e5418eb842e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:01:20.920346 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ad3f4-aa7e-4332-bdf3-e5418eb842e7 service cinder] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:01:20.932073 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1415/2865] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:01:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:01:30.926511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=54 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:01:34.201639 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ac05d-52a4-4d37-911c-a109453d9385 service swift] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:01:34.203049 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a81ac05d-52a4-4d37-911c-a109453d9385 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:01:34.260767 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ac05d-52a4-4d37-911c-a109453d9385 service swift] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:01:34.265565 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73802096-5170-4752-bbb3-797b092a5a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:01:34.266162 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73802096-5170-4752-bbb3-797b092a5a80 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:01:34.266435 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73802096-5170-4752-bbb3-797b092a5a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:01:34.266709 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73802096-5170-4752-bbb3-797b092a5a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:01:34.276336 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73802096-5170-4752-bbb3-797b092a5a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:01:34.278390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73802096-5170-4752-bbb3-797b092a5a80 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:01:34.287721 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1451/2866] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:01:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:01:44.283634 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=26 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:01:53.128254 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ad3f4-aa7e-4332-bdf3-e5418eb842e7 service cinder] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:01:53.137907 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ad3f4-aa7e-4332-bdf3-e5418eb842e7 service cinder] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:01:53.141811 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba7b46-e3d7-4c61-ab93-dccb0c743411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:01:53.142434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba7b46-e3d7-4c61-ab93-dccb0c743411 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:01:53.142713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba7b46-e3d7-4c61-ab93-dccb0c743411 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:01:53.142987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba7b46-e3d7-4c61-ab93-dccb0c743411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:01:53.144780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6cba7b46-e3d7-4c61-ab93-dccb0c743411 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:01:53.155633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba7b46-e3d7-4c61-ab93-dccb0c743411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:01:53.157017 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba7b46-e3d7-4c61-ab93-dccb0c743411 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:01:53.165412 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1416/2867] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:01:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:02:03.162391 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=6 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:02:04.898595 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73802096-5170-4752-bbb3-797b092a5a80 service neutron] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:04.899124 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-73802096-5170-4752-bbb3-797b092a5a80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:04.916723 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73802096-5170-4752-bbb3-797b092a5a80 service neutron] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:04.922347 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e2216-1937-4ffd-92ff-cac043707b3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:04.924057 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e2216-1937-4ffd-92ff-cac043707b3d service nova] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:04.924350 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e2216-1937-4ffd-92ff-cac043707b3d service nova] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:04.924627 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e2216-1937-4ffd-92ff-cac043707b3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:04.926899 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5d2e2216-1937-4ffd-92ff-cac043707b3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:04.940745 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e2216-1937-4ffd-92ff-cac043707b3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:04.942373 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e2216-1937-4ffd-92ff-cac043707b3d service nova] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:04.956536 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1452/2868] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:02:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:02:14.950778 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=6 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:02:17.318774 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba7b46-e3d7-4c61-ab93-dccb0c743411 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:17.327568 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba7b46-e3d7-4c61-ab93-dccb0c743411 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:17.331203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65fbf4-c96c-475a-a142-bc02948d2317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:17.331844 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65fbf4-c96c-475a-a142-bc02948d2317 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:17.332113 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65fbf4-c96c-475a-a142-bc02948d2317 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:17.332386 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65fbf4-c96c-475a-a142-bc02948d2317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:17.334114 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9e65fbf4-c96c-475a-a142-bc02948d2317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:17.397183 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65fbf4-c96c-475a-a142-bc02948d2317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:17.398604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65fbf4-c96c-475a-a142-bc02948d2317 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:17.408604 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1417/2869] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:02:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:02:23.344907 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e2216-1937-4ffd-92ff-cac043707b3d service nova] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:23.346439 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5d2e2216-1937-4ffd-92ff-cac043707b3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:23.410030 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65fbf4-c96c-475a-a142-bc02948d2317 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:23.412186 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9e65fbf4-c96c-475a-a142-bc02948d2317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:23.432379 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65fbf4-c96c-475a-a142-bc02948d2317 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:23.439798 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:23.440668 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:23.440943 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:23.441233 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] PATH_INFO: `/v3/users/ac4d918f549c41d3ae19e80cecefa95c` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:23.442381 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac4d918f549c41d3ae19e80cecefa95c)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:23.444661 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:23.449770 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e2216-1937-4ffd-92ff-cac043707b3d service nova] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:23.454961 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e689cc-ddce-4adf-828f-13a5f3a369bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:23.455573 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e689cc-ddce-4adf-828f-13a5f3a369bc service placement] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:23.455886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e689cc-ddce-4adf-828f-13a5f3a369bc service placement] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:23.456163 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e689cc-ddce-4adf-828f-13a5f3a369bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:23.458008 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f4e689cc-ddce-4adf-828f-13a5f3a369bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:23.470255 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e689cc-ddce-4adf-828f-13a5f3a369bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:23.472150 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e689cc-ddce-4adf-828f-13a5f3a369bc service placement] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:23.482871 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1453/2870] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:02:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:02:23.504523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac4d918f549c41d3ae19e80cecefa95c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:23.514495 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac4d918f549c41d3ae19e80cecefa95c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:23.515597 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac4d918f549c41d3ae19e80cecefa95c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:23.516451 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac4d918f549c41d3ae19e80cecefa95c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:23.529090 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac4d918f549c41d3ae19e80cecefa95c'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:23.543503 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1418/2871] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 15:02:23 2022] DELETE /identity/v3/users/ac4d918f549c41d3ae19e80cecefa95c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 15:02:23.721988 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e689cc-ddce-4adf-828f-13a5f3a369bc service placement] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:23.735232 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e689cc-ddce-4adf-828f-13a5f3a369bc service placement] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:23.794805 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:23.795443 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:23.795751 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:23.796029 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] PATH_INFO: `/v3/projects/d334ad6bd6194604a61ce4ca95149993` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:23.797146 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d334ad6bd6194604a61ce4ca95149993)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:23.799040 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:23.844746 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd334ad6bd6194604a61ce4ca95149993', 'request_id': 'req-784c6d2b-f491-45d1-8594-9b0a18acd8c0'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:23.850311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd334ad6bd6194604a61ce4ca95149993', 'request_id': 'req-784c6d2b-f491-45d1-8594-9b0a18acd8c0'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:23.859094 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd334ad6bd6194604a61ce4ca95149993', 'request_id': 'req-784c6d2b-f491-45d1-8594-9b0a18acd8c0'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:23.871354 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] The token cache is being invalidate because project d334ad6bd6194604a61ce4ca95149993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 15:02:23.871644 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:23.872327 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:23.873798 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1454/2872] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 15:02:23 2022] DELETE /identity/v3/projects/d334ad6bd6194604a61ce4ca95149993 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:02:23.877881 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:23.879324 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:23.892650 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade348f-55f7-4f86-b8c8-fac19a8414fb admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:23.949680 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:23.950313 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:23.950594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:23.950865 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] PATH_INFO: `/v3/users/8ef862d6f6674114b783993006f68258` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:23.952036 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef862d6f6674114b783993006f68258)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:23.953942 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:23.959274 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:02:24.002281 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef862d6f6674114b783993006f68258'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:24.009350 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef862d6f6674114b783993006f68258'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:24.010023 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef862d6f6674114b783993006f68258'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:24.010633 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef862d6f6674114b783993006f68258'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:24.022511 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef862d6f6674114b783993006f68258'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:24.034997 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1419/2873] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 15:02:23 2022] DELETE /identity/v3/users/8ef862d6f6674114b783993006f68258 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 15:02:24.209227 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:24.210718 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:24.224168 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c6d2b-f491-45d1-8594-9b0a18acd8c0 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:24.241545 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:02:24.284145 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb8d99-905d-41e6-9d49-49f66bc544ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:24.284772 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb8d99-905d-41e6-9d49-49f66bc544ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:24.285053 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb8d99-905d-41e6-9d49-49f66bc544ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:24.285324 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb8d99-905d-41e6-9d49-49f66bc544ba admin admin] PATH_INFO: `/v3/projects/8b91274d490e44c09edc6af01a0e13fb` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:24.286425 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb8d99-905d-41e6-9d49-49f66bc544ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b91274d490e44c09edc6af01a0e13fb)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:24.288390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb8d99-905d-41e6-9d49-49f66bc544ba admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:24.332917 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-1bcb8d99-905d-41e6-9d49-49f66bc544ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b91274d490e44c09edc6af01a0e13fb', 'request_id': 'req-1bcb8d99-905d-41e6-9d49-49f66bc544ba'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:24.338390 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-1bcb8d99-905d-41e6-9d49-49f66bc544ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b91274d490e44c09edc6af01a0e13fb', 'request_id': 'req-1bcb8d99-905d-41e6-9d49-49f66bc544ba'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:24.346306 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-1bcb8d99-905d-41e6-9d49-49f66bc544ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b91274d490e44c09edc6af01a0e13fb', 'request_id': 'req-1bcb8d99-905d-41e6-9d49-49f66bc544ba'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:24.357902 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-1bcb8d99-905d-41e6-9d49-49f66bc544ba admin admin] The token cache is being invalidate because project 8b91274d490e44c09edc6af01a0e13fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 15:02:24.358322 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-1bcb8d99-905d-41e6-9d49-49f66bc544ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:24.358860 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-1bcb8d99-905d-41e6-9d49-49f66bc544ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:24.360745 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1455/2874] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 15:02:24 2022] DELETE /identity/v3/projects/8b91274d490e44c09edc6af01a0e13fb => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:02:34.031629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:02:34.354819 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:02:39.716553 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:39.724695 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:39.813518 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d9127-01ff-4e0b-8814-2dcf5b56ece4 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:39.904456 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639150ab-6ee7-433d-a044-5fab0d910c23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:39.905091 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639150ab-6ee7-433d-a044-5fab0d910c23 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:39.905362 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639150ab-6ee7-433d-a044-5fab0d910c23 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:39.905629 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639150ab-6ee7-433d-a044-5fab0d910c23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:39.908311 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-639150ab-6ee7-433d-a044-5fab0d910c23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:39.921706 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639150ab-6ee7-433d-a044-5fab0d910c23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:39.923310 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639150ab-6ee7-433d-a044-5fab0d910c23 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:39.942905 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1420/2875] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:02:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:02:48.004997 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed70a10-9b58-4335-b2d0-831ca12331b2 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:48.005570 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed70a10-9b58-4335-b2d0-831ca12331b2 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:48.005681 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed70a10-9b58-4335-b2d0-831ca12331b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:48.064067 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-5ed70a10-9b58-4335-b2d0-831ca12331b2 None None] MFA Rules not processed for user `cbb13837e1694d7aa928ac855e21d1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:02:48.065579 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-5ed70a10-9b58-4335-b2d0-831ca12331b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:48.145511 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1456/2876] 10.222.0.36 () {58 vars in 967 bytes} [Thu Aug 18 15:02:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:02:48.201715 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639150ab-6ee7-433d-a044-5fab0d910c23 service nova] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:48.203157 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-639150ab-6ee7-433d-a044-5fab0d910c23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:48.315931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639150ab-6ee7-433d-a044-5fab0d910c23 service nova] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:48.315931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b88f1d-721c-4229-a1a3-9439576abd88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f578727b96564c3db6b5a1e6cac109d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:48.315931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b88f1d-721c-4229-a1a3-9439576abd88 service swift] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:48.315931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b88f1d-721c-4229-a1a3-9439576abd88 service swift] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:48.316366 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b88f1d-721c-4229-a1a3-9439576abd88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:48.318584 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=100 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:02:48.323697 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b88f1d-721c-4229-a1a3-9439576abd88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:48.325269 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b88f1d-721c-4229-a1a3-9439576abd88 service swift] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:48.333466 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1421/2877] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Aug 18 15:02:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 18 15:02:50.984681 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d6e2a-a457-483c-9941-484292ed99ba None None] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:50.988315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d6e2a-a457-483c-9941-484292ed99ba None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:50.991610 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d6e2a-a457-483c-9941-484292ed99ba None None] PATH_INFO: `/` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:50.993696 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1457/2878] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 15:02:50 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 15:02:51.004007 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215addd-8d0f-4b1f-b36d-6f13bf74cd87 None None] REQUEST_METHOD: `POST` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:51.004579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215addd-8d0f-4b1f-b36d-6f13bf74cd87 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:51.004940 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215addd-8d0f-4b1f-b36d-6f13bf74cd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:51.056480 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.auth.core [None req-9215addd-8d0f-4b1f-b36d-6f13bf74cd87 None None] MFA Rules not processed for user `00f209d8426a49acaca7e9d00a1b9ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=93179) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:02:51.057737 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9215addd-8d0f-4b1f-b36d-6f13bf74cd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:51.123589 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1422/2879] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:02:51.136483 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d6e2a-a457-483c-9941-484292ed99ba None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:51.138624 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f84d6e2a-a457-483c-9941-484292ed99ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:51.163502 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d6e2a-a457-483c-9941-484292ed99ba None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:51.169601 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef770487-0447-47a7-8a97-965e21500bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:51.170291 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef770487-0447-47a7-8a97-965e21500bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:51.170647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef770487-0447-47a7-8a97-965e21500bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:51.171037 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef770487-0447-47a7-8a97-965e21500bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:51.178791 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef770487-0447-47a7-8a97-965e21500bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:51.180429 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef770487-0447-47a7-8a97-965e21500bd0 service neutron] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:51.189688 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1458/2880] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:02:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 15:02:52.313622 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791bb74-9858-46eb-a44f-12bfcc2ed014 None None] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:52.314209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791bb74-9858-46eb-a44f-12bfcc2ed014 None None] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:52.314359 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791bb74-9858-46eb-a44f-12bfcc2ed014 None None] PATH_INFO: `/` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:52.316351 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1423/2881] 10.222.0.36 () {54 vars in 921 bytes} [Thu Aug 18 15:02:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 15:02:52.324893 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d5f26-ef36-42db-b235-a05494d51929 None None] REQUEST_METHOD: `POST` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:52.325607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d5f26-ef36-42db-b235-a05494d51929 None None] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:52.325607 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d5f26-ef36-42db-b235-a05494d51929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:52.384102 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.auth.core [None req-383d5f26-ef36-42db-b235-a05494d51929 None None] MFA Rules not processed for user `3f3efa9f3eb24864943b68d241435817`. Rule list: `[]` (Enabled: `True`). {{(pid=93178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 15:02:52.385420 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-383d5f26-ef36-42db-b235-a05494d51929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:52.401092 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1459/2882] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Aug 18 15:02:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 15:02:52.414163 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791bb74-9858-46eb-a44f-12bfcc2ed014 None None] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:52.416948 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6791bb74-9858-46eb-a44f-12bfcc2ed014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:52.450009 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791bb74-9858-46eb-a44f-12bfcc2ed014 None None] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:52.457791 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96848d-25a2-4f40-889f-b40704a1ae11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f209d8426a49acaca7e9d00a1b9ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:52.458707 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96848d-25a2-4f40-889f-b40704a1ae11 service neutron] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:52.459123 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96848d-25a2-4f40-889f-b40704a1ae11 service neutron] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:52.459555 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96848d-25a2-4f40-889f-b40704a1ae11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:52.476799 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96848d-25a2-4f40-889f-b40704a1ae11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:52.479722 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96848d-25a2-4f40-889f-b40704a1ae11 service neutron] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:52.500477 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1424/2883] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:02:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:02:52.581930 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d5f26-ef36-42db-b235-a05494d51929 None None] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:52.583886 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-383d5f26-ef36-42db-b235-a05494d51929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:52.669758 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d5f26-ef36-42db-b235-a05494d51929 None None] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:52.675122 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cf576-f6ab-4e5e-8784-4f29b63251e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:52.675760 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cf576-f6ab-4e5e-8784-4f29b63251e3 service placement] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:52.676039 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cf576-f6ab-4e5e-8784-4f29b63251e3 service placement] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:52.676311 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cf576-f6ab-4e5e-8784-4f29b63251e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:52.678077 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-587cf576-f6ab-4e5e-8784-4f29b63251e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:52.690286 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cf576-f6ab-4e5e-8784-4f29b63251e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:52.691900 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cf576-f6ab-4e5e-8784-4f29b63251e3 service placement] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:52.702113 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1460/2884] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:02:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:02:54.952655 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96848d-25a2-4f40-889f-b40704a1ae11 service neutron] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:54.954651 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-1c96848d-25a2-4f40-889f-b40704a1ae11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:54.981371 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96848d-25a2-4f40-889f-b40704a1ae11 service neutron] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:55.002514 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd0118-384c-4cd9-827c-c2cba17ff7a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3efa9f3eb24864943b68d241435817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:55.003523 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd0118-384c-4cd9-827c-c2cba17ff7a3 service nova] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:55.004110 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd0118-384c-4cd9-827c-c2cba17ff7a3 service nova] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:55.004856 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd0118-384c-4cd9-827c-c2cba17ff7a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:55.008056 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-8ddd0118-384c-4cd9-827c-c2cba17ff7a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:55.038044 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd0118-384c-4cd9-827c-c2cba17ff7a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:55.040077 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd0118-384c-4cd9-827c-c2cba17ff7a3 service nova] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:55.074672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1425/2885] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:02:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 15:02:56.229501 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cf576-f6ab-4e5e-8784-4f29b63251e3 service placement] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:56.234755 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-587cf576-f6ab-4e5e-8784-4f29b63251e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:56.288522 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=100 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:02:56.339408 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cf576-f6ab-4e5e-8784-4f29b63251e3 service placement] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:56.345555 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:56.346219 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:56.346500 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:56.346780 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] PATH_INFO: `/v3/users/57632501742348109ed397d84c71510a` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:56.348020 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] RBAC: Authorizing `identity:delete_user(user_id=57632501742348109ed397d84c71510a)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:56.350089 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:56.407250 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57632501742348109ed397d84c71510a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:56.414393 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57632501742348109ed397d84c71510a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:56.415098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57632501742348109ed397d84c71510a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:56.427959 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57632501742348109ed397d84c71510a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:56.439241 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57632501742348109ed397d84c71510a'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:56.448977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1461/2886] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 15:02:56 2022] DELETE /identity/v3/users/57632501742348109ed397d84c71510a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 15:02:56.654441 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd0118-384c-4cd9-827c-c2cba17ff7a3 service nova] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:56.667192 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd0118-384c-4cd9-827c-c2cba17ff7a3 service nova] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:56.729364 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:56.730020 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:56.730319 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:56.730637 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] PATH_INFO: `/v3/projects/1f169db98a4549109d83c18d108b8282` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:56.732107 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f169db98a4549109d83c18d108b8282)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:56.734341 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:56.789343 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f169db98a4549109d83c18d108b8282', 'request_id': 'req-51bac03e-e2f9-4762-907e-d7dd8a918f81'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:56.795986 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f169db98a4549109d83c18d108b8282', 'request_id': 'req-51bac03e-e2f9-4762-907e-d7dd8a918f81'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:56.804223 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f169db98a4549109d83c18d108b8282', 'request_id': 'req-51bac03e-e2f9-4762-907e-d7dd8a918f81'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:56.806614 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:02:56.820275 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] The token cache is being invalidate because project 1f169db98a4549109d83c18d108b8282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 15:02:56.820545 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:56.821321 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:56.823906 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1426/2887] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 15:02:56 2022] DELETE /identity/v3/projects/1f169db98a4549109d83c18d108b8282 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:02:56.831016 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:56.835835 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:56.851351 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18731591-2931-4ea0-a5d1-5dc5485b103d admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:56.915769 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:56.916535 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:56.916977 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:56.917360 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] PATH_INFO: `/v3/users/8951ef845ca74c2cb4df80d364534c6d` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:56.918932 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8951ef845ca74c2cb4df80d364534c6d)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:56.922067 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:56.980376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8951ef845ca74c2cb4df80d364534c6d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:56.982712 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:02:56.989471 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8951ef845ca74c2cb4df80d364534c6d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:56.990728 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8951ef845ca74c2cb4df80d364534c6d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:57.004263 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8951ef845ca74c2cb4df80d364534c6d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:57.013512 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8951ef845ca74c2cb4df80d364534c6d'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:57.020461 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1462/2888] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 15:02:56 2022] DELETE /identity/v3/users/8951ef845ca74c2cb4df80d364534c6d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:02:57.212708 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:02:57.214243 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:02:57.253612 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bac03e-e2f9-4762-907e-d7dd8a918f81 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:02:57.338910 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:02:57.339896 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:02:57.340433 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:02:57.340730 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] PATH_INFO: `/v3/projects/59bf5e892b4043db9628d6f3b4149aab` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:02:57.342780 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59bf5e892b4043db9628d6f3b4149aab)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:02:57.344147 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:02:57.398903 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59bf5e892b4043db9628d6f3b4149aab', 'request_id': 'req-9a3d940a-ed7a-47f0-897f-b8be578bbff6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:57.405026 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59bf5e892b4043db9628d6f3b4149aab', 'request_id': 'req-9a3d940a-ed7a-47f0-897f-b8be578bbff6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:57.418579 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59bf5e892b4043db9628d6f3b4149aab', 'request_id': 'req-9a3d940a-ed7a-47f0-897f-b8be578bbff6'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:57.430275 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] The token cache is being invalidate because project 59bf5e892b4043db9628d6f3b4149aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 15:02:57.430594 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:57.431734 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:02:57.433209 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1427/2889] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 15:02:57 2022] DELETE /identity/v3/projects/59bf5e892b4043db9628d6f3b4149aab => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:03:07.017083 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=11,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:03:07.427272 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:04:35.012878 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:04:35.014986 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:04:35.100852 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb7d20-7c59-4d1c-b439-2fd35fed3b6c admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:04:35.165385 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d11507-1139-4174-9924-33f7ecc78fce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:04:35.166098 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d11507-1139-4174-9924-33f7ecc78fce service placement] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:04:35.166560 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d11507-1139-4174-9924-33f7ecc78fce service placement] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:04:35.166944 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d11507-1139-4174-9924-33f7ecc78fce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:04:35.169376 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a0d11507-1139-4174-9924-33f7ecc78fce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:04:35.181559 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d11507-1139-4174-9924-33f7ecc78fce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:04:35.183315 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d11507-1139-4174-9924-33f7ecc78fce service placement] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:04:35.194772 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1463/2890] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:04:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:04:38.223444 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:04:38.225777 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:04:38.243426 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d940a-ed7a-47f0-897f-b8be578bbff6 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:04:38.248515 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ede1e-37b2-44f9-8471-4ea082491e04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:04:38.249120 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ede1e-37b2-44f9-8471-4ea082491e04 service placement] REQUEST_METHOD: `GET` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:04:38.249601 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ede1e-37b2-44f9-8471-4ea082491e04 service placement] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:04:38.249876 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ede1e-37b2-44f9-8471-4ea082491e04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:04:38.251713 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-5d4ede1e-37b2-44f9-8471-4ea082491e04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:04:38.263673 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ede1e-37b2-44f9-8471-4ea082491e04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:04:38.265627 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ede1e-37b2-44f9-8471-4ea082491e04 service placement] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:04:38.278051 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1428/2891] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:04:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:04:38.292344 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d11507-1139-4174-9924-33f7ecc78fce service placement] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:04:38.295024 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-a0d11507-1139-4174-9924-33f7ecc78fce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:04:38.363139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d11507-1139-4174-9924-33f7ecc78fce service placement] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:04:38.367647 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:04:38.368310 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:04:38.368585 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:04:38.368851 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] PATH_INFO: `/v3/users/1e3a4f725f3d41edbb7c162a8760bf8b` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:04:38.369973 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e3a4f725f3d41edbb7c162a8760bf8b)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:04:38.371915 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:04:38.409415 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:04:38.419626 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e3a4f725f3d41edbb7c162a8760bf8b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.426548 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e3a4f725f3d41edbb7c162a8760bf8b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.427212 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e3a4f725f3d41edbb7c162a8760bf8b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.447343 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e3a4f725f3d41edbb7c162a8760bf8b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.455764 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e3a4f725f3d41edbb7c162a8760bf8b'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.462199 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1464/2892] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 15:04:38 2022] DELETE /identity/v3/users/1e3a4f725f3d41edbb7c162a8760bf8b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:04:38.647203 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ede1e-37b2-44f9-8471-4ea082491e04 service placement] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:04:38.658852 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ede1e-37b2-44f9-8471-4ea082491e04 service placement] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:04:38.722499 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:04:38.723129 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:04:38.723403 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:04:38.723670 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] PATH_INFO: `/v3/projects/379b08d3cf834ca9b891abd6ba0f14ee` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:04:38.724824 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=379b08d3cf834ca9b891abd6ba0f14ee)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:04:38.726714 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:04:38.774104 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '379b08d3cf834ca9b891abd6ba0f14ee', 'request_id': 'req-6fb3e951-e37c-4fce-a1b3-c5d391b53622'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.779388 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '379b08d3cf834ca9b891abd6ba0f14ee', 'request_id': 'req-6fb3e951-e37c-4fce-a1b3-c5d391b53622'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.788022 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '379b08d3cf834ca9b891abd6ba0f14ee', 'request_id': 'req-6fb3e951-e37c-4fce-a1b3-c5d391b53622'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.799672 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] The token cache is being invalidate because project 379b08d3cf834ca9b891abd6ba0f14ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 15:04:38.799987 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.800623 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.802154 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1429/2893] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 15:04:38 2022] DELETE /identity/v3/projects/379b08d3cf834ca9b891abd6ba0f14ee => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:04:38.808202 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:04:38.809623 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:04:38.824309 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81013ba-5bf6-4a98-b38c-5402f3c10c8a admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:04:38.881248 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:04:38.881865 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:04:38.882142 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:04:38.882412 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] PATH_INFO: `/v3/users/e105e9d96c7d4cc296745ee791efaa2e` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:04:38.883530 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e105e9d96c7d4cc296745ee791efaa2e)` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:04:38.885557 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:04:38.935572 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e105e9d96c7d4cc296745ee791efaa2e'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.943088 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e105e9d96c7d4cc296745ee791efaa2e'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.943831 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e105e9d96c7d4cc296745ee791efaa2e'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.956139 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e105e9d96c7d4cc296745ee791efaa2e'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.963299 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.notifications [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e105e9d96c7d4cc296745ee791efaa2e'} {{(pid=93178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:38.969661 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1465/2894] 10.222.0.36 () {58 vars in 1224 bytes} [Thu Aug 18 15:04:38 2022] DELETE /identity/v3/users/e105e9d96c7d4cc296745ee791efaa2e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 15:04:39.149573 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] Authenticating user token {{(pid=93179) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:04:39.151082 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.fernet_utils [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93179) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:04:39.163768 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3e951-e37c-4fce-a1b3-c5d391b53622 admin admin] Validating token access rules against request {{(pid=93179) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:04:39.222535 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5064296cd28f403599a02675cb0c7103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa81e49fae72473b99b445418d4fe5de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93179) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:04:39.223159 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:04:39.223434 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244 admin admin] SCRIPT_NAME: `/identity` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:04:39.223847 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244 admin admin] PATH_INFO: `/v3/projects/80b2b237be46413fa8742f0662512149` {{(pid=93179) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:04:39.224961 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b2b237be46413fa8742f0662512149)` {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:04:39.226854 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244 admin admin] RBAC: Authorization granted {{(pid=93179) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:04:39.247991 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:04:39.277557 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b2b237be46413fa8742f0662512149', 'request_id': 'req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:39.284079 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b2b237be46413fa8742f0662512149', 'request_id': 'req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:39.291931 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b2b237be46413fa8742f0662512149', 'request_id': 'req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244'} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:39.305282 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244 admin admin] The token cache is being invalidate because project 80b2b237be46413fa8742f0662512149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93179) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 15:04:39.305635 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:39.306333 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG keystone.notifications [None req-73cb07b5-1e8f-4dc8-ba49-5819cee8e244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93179) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 15:04:39.307982 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: [pid: 93179|app: 0|req: 1430/2895] 10.222.0.36 () {58 vars in 1233 bytes} [Thu Aug 18 15:04:39 2022] DELETE /identity/v3/projects/80b2b237be46413fa8742f0662512149 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 15:04:48.965762 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=69,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:04:49.302000 dv-cinder-852208-7-iscsi devstack@keystone.service[93179]: DEBUG dbcounter [-] [93179] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93179) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 15:09:43.176725 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] Authenticating user token {{(pid=93178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 15:09:43.178167 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:09:43.327467 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80dd8df-841b-4830-832c-b028bd274839 admin admin] Validating token access rules against request {{(pid=93178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 15:09:43.389880 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1154999-b9cc-4a64-b8d4-2f7b14e3d997 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db581b10d78a4f9eae33fadce0d92b0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12f9875f45a84e92a848b8fc1636d6c6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 15:09:43.390487 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1154999-b9cc-4a64-b8d4-2f7b14e3d997 service placement] REQUEST_METHOD: `GET` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 15:09:43.391017 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1154999-b9cc-4a64-b8d4-2f7b14e3d997 service placement] SCRIPT_NAME: `/identity` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 15:09:43.391284 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1154999-b9cc-4a64-b8d4-2f7b14e3d997 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 15:09:43.393126 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.fernet_utils [None req-f1154999-b9cc-4a64-b8d4-2f7b14e3d997 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 15:09:43.404583 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1154999-b9cc-4a64-b8d4-2f7b14e3d997 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 15:09:43.406154 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1154999-b9cc-4a64-b8d4-2f7b14e3d997 service placement] RBAC: Authorization granted {{(pid=93178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 15:09:43.416258 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: [pid: 93178|app: 0|req: 1466/2896] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Aug 18 15:09:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 15:09:53.416519 dv-cinder-852208-7-iscsi devstack@keystone.service[93178]: DEBUG dbcounter [-] [93178] Writing DB stats keystone:SELECT=89 {{(pid=93178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}